2023-07-16 21:15:26,932 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511 2023-07-16 21:15:26,952 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics timeout: 13 mins 2023-07-16 21:15:26,970 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-16 21:15:26,971 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5, deleteOnExit=true 2023-07-16 21:15:26,971 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-16 21:15:26,971 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/test.cache.data in system properties and HBase conf 2023-07-16 21:15:26,972 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.tmp.dir in system properties and HBase conf 2023-07-16 21:15:26,973 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir in system properties and HBase conf 2023-07-16 21:15:26,973 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-16 21:15:26,974 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-16 21:15:26,974 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-16 21:15:27,108 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-16 21:15:27,604 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-16 21:15:27,611 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-16 21:15:27,611 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-16 21:15:27,612 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-16 21:15:27,612 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-16 21:15:27,613 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-16 21:15:27,613 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-16 21:15:27,613 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-16 21:15:27,614 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-16 21:15:27,615 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-16 21:15:27,615 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/nfs.dump.dir in system properties and HBase conf 2023-07-16 21:15:27,616 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir in system properties and HBase conf 2023-07-16 21:15:27,617 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-16 21:15:27,617 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-16 21:15:27,618 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-16 21:15:28,175 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-16 21:15:28,179 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-16 21:15:28,480 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-16 21:15:28,681 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-16 21:15:28,708 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-16 21:15:28,744 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-16 21:15:28,788 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/Jetty_localhost_35273_hdfs____ua30d2/webapp 2023-07-16 21:15:28,951 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35273 2023-07-16 21:15:28,961 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-16 21:15:28,961 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-16 21:15:29,440 WARN [Listener at localhost/35261] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-16 21:15:29,534 WARN [Listener at localhost/35261] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-16 21:15:29,554 WARN [Listener at localhost/35261] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-16 21:15:29,561 INFO [Listener at localhost/35261] log.Slf4jLog(67): jetty-6.1.26 2023-07-16 21:15:29,565 INFO [Listener at localhost/35261] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/Jetty_localhost_34535_datanode____wau8n1/webapp 2023-07-16 21:15:29,665 INFO [Listener at localhost/35261] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34535 2023-07-16 21:15:30,126 WARN [Listener at localhost/43649] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-16 21:15:30,186 WARN [Listener at localhost/43649] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-16 21:15:30,196 WARN [Listener at localhost/43649] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-16 21:15:30,200 INFO [Listener at localhost/43649] log.Slf4jLog(67): jetty-6.1.26 2023-07-16 21:15:30,205 INFO [Listener at localhost/43649] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/Jetty_localhost_34911_datanode____q3kq9r/webapp 2023-07-16 21:15:30,331 INFO [Listener at localhost/43649] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34911 2023-07-16 21:15:30,345 WARN [Listener at localhost/43177] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-16 21:15:30,393 WARN [Listener at localhost/43177] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-16 21:15:30,403 WARN [Listener at localhost/43177] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-16 21:15:30,406 INFO [Listener at localhost/43177] log.Slf4jLog(67): jetty-6.1.26 2023-07-16 21:15:30,420 INFO [Listener at localhost/43177] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/Jetty_localhost_33577_datanode____.2ek17o/webapp 2023-07-16 21:15:30,558 INFO [Listener at localhost/43177] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33577 2023-07-16 21:15:30,578 WARN [Listener at localhost/43605] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-16 21:15:30,746 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x598b83d10542e53e: Processing first storage report for DS-d9831650-bcbe-4b26-9a48-5cc502a954ad from datanode 9f822f71-4eeb-4eb3-b4c1-9f5b5890a878 2023-07-16 21:15:30,748 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x598b83d10542e53e: from storage DS-d9831650-bcbe-4b26-9a48-5cc502a954ad node DatanodeRegistration(127.0.0.1:36325, datanodeUuid=9f822f71-4eeb-4eb3-b4c1-9f5b5890a878, infoPort=40811, infoSecurePort=0, ipcPort=43605, storageInfo=lv=-57;cid=testClusterID;nsid=641840174;c=1689542128252), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-07-16 21:15:30,748 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc76fa00b857df7d3: Processing first storage report for DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1 from datanode 6e68cdb2-c990-4136-ad44-2fec182ce26b 2023-07-16 21:15:30,748 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc76fa00b857df7d3: from storage DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1 node DatanodeRegistration(127.0.0.1:46257, datanodeUuid=6e68cdb2-c990-4136-ad44-2fec182ce26b, infoPort=38369, infoSecurePort=0, ipcPort=43649, storageInfo=lv=-57;cid=testClusterID;nsid=641840174;c=1689542128252), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-16 21:15:30,748 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xbff1189f776c575f: Processing first storage report for DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00 from datanode 86c0afe4-f4d7-43f2-9612-8fecb0477082 2023-07-16 21:15:30,748 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xbff1189f776c575f: from storage DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00 node DatanodeRegistration(127.0.0.1:43471, datanodeUuid=86c0afe4-f4d7-43f2-9612-8fecb0477082, infoPort=44555, infoSecurePort=0, ipcPort=43177, storageInfo=lv=-57;cid=testClusterID;nsid=641840174;c=1689542128252), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-16 21:15:30,749 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x598b83d10542e53e: Processing first storage report for DS-184a28f1-3fb7-4063-b5a1-37508b2be971 from datanode 9f822f71-4eeb-4eb3-b4c1-9f5b5890a878 2023-07-16 21:15:30,749 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x598b83d10542e53e: from storage DS-184a28f1-3fb7-4063-b5a1-37508b2be971 node DatanodeRegistration(127.0.0.1:36325, datanodeUuid=9f822f71-4eeb-4eb3-b4c1-9f5b5890a878, infoPort=40811, infoSecurePort=0, ipcPort=43605, storageInfo=lv=-57;cid=testClusterID;nsid=641840174;c=1689542128252), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-16 21:15:30,749 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc76fa00b857df7d3: Processing first storage report for DS-798445b8-3f34-4d5f-b052-d4b94064fa60 from datanode 6e68cdb2-c990-4136-ad44-2fec182ce26b 2023-07-16 21:15:30,749 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc76fa00b857df7d3: from storage DS-798445b8-3f34-4d5f-b052-d4b94064fa60 node DatanodeRegistration(127.0.0.1:46257, datanodeUuid=6e68cdb2-c990-4136-ad44-2fec182ce26b, infoPort=38369, infoSecurePort=0, ipcPort=43649, storageInfo=lv=-57;cid=testClusterID;nsid=641840174;c=1689542128252), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-16 21:15:30,749 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xbff1189f776c575f: Processing first storage report for DS-232656c2-577f-4904-94c7-34873007b135 from datanode 86c0afe4-f4d7-43f2-9612-8fecb0477082 2023-07-16 21:15:30,750 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xbff1189f776c575f: from storage DS-232656c2-577f-4904-94c7-34873007b135 node DatanodeRegistration(127.0.0.1:43471, datanodeUuid=86c0afe4-f4d7-43f2-9612-8fecb0477082, infoPort=44555, infoSecurePort=0, ipcPort=43177, storageInfo=lv=-57;cid=testClusterID;nsid=641840174;c=1689542128252), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-16 21:15:31,001 DEBUG [Listener at localhost/43605] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511 2023-07-16 21:15:31,096 INFO [Listener at localhost/43605] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5/zookeeper_0, clientPort=63917, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-16 21:15:31,112 INFO [Listener at localhost/43605] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=63917 2023-07-16 21:15:31,121 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:31,123 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:31,838 INFO [Listener at localhost/43605] util.FSUtils(471): Created version file at hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb with version=8 2023-07-16 21:15:31,838 INFO [Listener at localhost/43605] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/hbase-staging 2023-07-16 21:15:31,847 DEBUG [Listener at localhost/43605] hbase.LocalHBaseCluster(134): Setting Master Port to random. 2023-07-16 21:15:31,847 DEBUG [Listener at localhost/43605] hbase.LocalHBaseCluster(141): Setting RegionServer Port to random. 2023-07-16 21:15:31,847 DEBUG [Listener at localhost/43605] hbase.LocalHBaseCluster(151): Setting RS InfoServer Port to random. 2023-07-16 21:15:31,847 DEBUG [Listener at localhost/43605] hbase.LocalHBaseCluster(159): Setting Master InfoServer Port to random. 2023-07-16 21:15:32,216 INFO [Listener at localhost/43605] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-16 21:15:32,785 INFO [Listener at localhost/43605] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:15:32,828 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:32,829 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:32,829 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:15:32,829 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:32,829 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:15:32,983 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:15:33,076 DEBUG [Listener at localhost/43605] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-16 21:15:33,202 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41499 2023-07-16 21:15:33,215 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:33,217 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:33,245 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=master:41499 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:15:33,301 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:414990x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:15:33,305 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:41499-0x1016ffd80780000 connected 2023-07-16 21:15:33,346 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-16 21:15:33,347 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:15:33,351 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:15:33,363 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41499 2023-07-16 21:15:33,364 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41499 2023-07-16 21:15:33,365 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41499 2023-07-16 21:15:33,366 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41499 2023-07-16 21:15:33,366 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41499 2023-07-16 21:15:33,399 INFO [Listener at localhost/43605] log.Log(170): Logging initialized @7263ms to org.apache.hbase.thirdparty.org.eclipse.jetty.util.log.Slf4jLog 2023-07-16 21:15:33,547 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:15:33,548 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:15:33,549 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:15:33,551 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context master 2023-07-16 21:15:33,551 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:15:33,552 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:15:33,556 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:15:33,632 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 43223 2023-07-16 21:15:33,634 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:15:33,685 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:33,689 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@687841de{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:15:33,690 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:33,690 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@2356c1aa{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:15:33,904 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:15:33,918 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:15:33,918 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:15:33,921 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-16 21:15:33,931 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:33,958 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@78cf9e5c{master,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-43223-hbase-server-2_4_18-SNAPSHOT_jar-_-any-771142757187536015/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/master} 2023-07-16 21:15:33,969 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@5ce4cc97{HTTP/1.1, (http/1.1)}{0.0.0.0:43223} 2023-07-16 21:15:33,969 INFO [Listener at localhost/43605] server.Server(415): Started @7834ms 2023-07-16 21:15:33,973 INFO [Listener at localhost/43605] master.HMaster(444): hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb, hbase.cluster.distributed=false 2023-07-16 21:15:34,044 INFO [Listener at localhost/43605] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:15:34,045 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:34,045 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:34,045 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:15:34,045 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:34,045 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:15:34,051 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:15:34,054 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36993 2023-07-16 21:15:34,056 INFO [Listener at localhost/43605] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-16 21:15:34,063 DEBUG [Listener at localhost/43605] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-16 21:15:34,064 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:34,066 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:34,067 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36993 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:15:34,071 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:369930x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:15:34,073 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36993-0x1016ffd80780001 connected 2023-07-16 21:15:34,073 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-16 21:15:34,075 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:15:34,075 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:15:34,076 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36993 2023-07-16 21:15:34,076 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36993 2023-07-16 21:15:34,077 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36993 2023-07-16 21:15:34,077 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36993 2023-07-16 21:15:34,077 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36993 2023-07-16 21:15:34,080 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:15:34,080 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:15:34,080 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:15:34,081 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-16 21:15:34,081 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:15:34,081 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:15:34,081 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:15:34,083 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 45735 2023-07-16 21:15:34,083 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:15:34,087 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:34,087 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@69738833{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:15:34,088 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:34,088 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@7ea3cad5{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:15:34,212 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:15:34,214 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:15:34,214 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:15:34,214 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-16 21:15:34,216 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:34,219 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@5a679f9e{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-45735-hbase-server-2_4_18-SNAPSHOT_jar-_-any-4115428025028425592/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:15:34,220 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@25eab107{HTTP/1.1, (http/1.1)}{0.0.0.0:45735} 2023-07-16 21:15:34,221 INFO [Listener at localhost/43605] server.Server(415): Started @8085ms 2023-07-16 21:15:34,234 INFO [Listener at localhost/43605] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:15:34,234 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:34,234 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:34,235 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:15:34,235 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:34,235 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:15:34,235 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:15:34,237 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45229 2023-07-16 21:15:34,237 INFO [Listener at localhost/43605] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-16 21:15:34,238 DEBUG [Listener at localhost/43605] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-16 21:15:34,239 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:34,240 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:34,241 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45229 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:15:34,246 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:452290x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:15:34,247 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:45229-0x1016ffd80780002 connected 2023-07-16 21:15:34,247 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-16 21:15:34,248 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:15:34,249 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:15:34,249 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45229 2023-07-16 21:15:34,250 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45229 2023-07-16 21:15:34,251 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45229 2023-07-16 21:15:34,251 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45229 2023-07-16 21:15:34,251 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45229 2023-07-16 21:15:34,255 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:15:34,255 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:15:34,255 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:15:34,256 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-16 21:15:34,256 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:15:34,256 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:15:34,256 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:15:34,257 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 45869 2023-07-16 21:15:34,257 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:15:34,261 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:34,261 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@425bb8be{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:15:34,262 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:34,262 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@38834d1b{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:15:34,418 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:15:34,420 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:15:34,420 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:15:34,421 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-16 21:15:34,422 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:34,423 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@7c8e2ea{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-45869-hbase-server-2_4_18-SNAPSHOT_jar-_-any-7932878267613358111/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:15:34,424 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@12e45025{HTTP/1.1, (http/1.1)}{0.0.0.0:45869} 2023-07-16 21:15:34,424 INFO [Listener at localhost/43605] server.Server(415): Started @8289ms 2023-07-16 21:15:34,438 INFO [Listener at localhost/43605] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:15:34,439 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:34,439 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:34,439 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:15:34,439 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:34,440 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:15:34,440 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:15:34,442 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33511 2023-07-16 21:15:34,442 INFO [Listener at localhost/43605] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-16 21:15:34,444 DEBUG [Listener at localhost/43605] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-16 21:15:34,445 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:34,446 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:34,447 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33511 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:15:34,454 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:335110x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:15:34,454 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:335110x0, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-16 21:15:34,458 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:335110x0, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:15:34,459 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:335110x0, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:15:34,461 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:33511-0x1016ffd80780003 connected 2023-07-16 21:15:34,462 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33511 2023-07-16 21:15:34,463 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33511 2023-07-16 21:15:34,474 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33511 2023-07-16 21:15:34,475 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33511 2023-07-16 21:15:34,475 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33511 2023-07-16 21:15:34,478 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:15:34,478 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:15:34,479 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:15:34,479 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-16 21:15:34,480 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:15:34,480 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:15:34,480 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:15:34,481 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 42947 2023-07-16 21:15:34,481 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:15:34,500 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:34,501 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@50d1e028{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:15:34,501 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:34,501 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@6ae3c253{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:15:34,623 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:15:34,624 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:15:34,625 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:15:34,625 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 600000ms 2023-07-16 21:15:34,626 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:34,627 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@4ad0acc{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-42947-hbase-server-2_4_18-SNAPSHOT_jar-_-any-785865430246237267/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:15:34,629 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@5a58295b{HTTP/1.1, (http/1.1)}{0.0.0.0:42947} 2023-07-16 21:15:34,629 INFO [Listener at localhost/43605] server.Server(415): Started @8493ms 2023-07-16 21:15:34,636 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:15:34,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.AbstractConnector(333): Started ServerConnector@30d25553{HTTP/1.1, (http/1.1)}{0.0.0.0:34051} 2023-07-16 21:15:34,651 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.Server(415): Started @8515ms 2023-07-16 21:15:34,651 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,41499,1689542132027 2023-07-16 21:15:34,664 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-16 21:15:34,666 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,41499,1689542132027 2023-07-16 21:15:34,693 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-16 21:15:34,693 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-16 21:15:34,693 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-16 21:15:34,694 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:15:34,696 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-16 21:15:34,699 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-16 21:15:34,699 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-16 21:15:34,702 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,41499,1689542132027 from backup master directory 2023-07-16 21:15:34,714 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,41499,1689542132027 2023-07-16 21:15:34,714 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-16 21:15:34,715 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:15:34,716 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,41499,1689542132027 2023-07-16 21:15:34,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-16 21:15:34,759 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-16 21:15:34,894 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/hbase.id with ID: bc8d5a8a-d305-489f-bc1d-b95b98296a6c 2023-07-16 21:15:34,947 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:34,967 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:15:35,043 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x084e601d to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:15:35,068 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@e81a343, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:15:35,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-16 21:15:35,095 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-16 21:15:35,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-07-16 21:15:35,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-07-16 21:15:35,118 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-16 21:15:35,122 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-16 21:15:35,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:15:35,164 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store-tmp 2023-07-16 21:15:35,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:35,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-16 21:15:35,208 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:15:35,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:15:35,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-16 21:15:35,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:15:35,208 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:15:35,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-16 21:15:35,210 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,41499,1689542132027 2023-07-16 21:15:35,234 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41499%2C1689542132027, suffix=, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,41499,1689542132027, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/oldWALs, maxLogs=10 2023-07-16 21:15:35,314 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:15:35,314 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:15:35,314 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:15:35,325 DEBUG [RS-EventLoopGroup-5-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-16 21:15:35,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,41499,1689542132027/jenkins-hbase4.apache.org%2C41499%2C1689542132027.1689542135245 2023-07-16 21:15:35,419 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK], DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK], DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK]] 2023-07-16 21:15:35,420 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:15:35,421 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:35,425 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-16 21:15:35,427 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-16 21:15:35,513 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-16 21:15:35,522 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-16 21:15:35,558 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-16 21:15:35,573 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:35,578 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-16 21:15:35,580 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-16 21:15:35,597 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-16 21:15:35,601 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:35,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=9753473120, jitterRate=-0.0916370302438736}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:35,602 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-16 21:15:35,603 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-16 21:15:35,626 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-16 21:15:35,626 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-16 21:15:35,629 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-16 21:15:35,630 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-16 21:15:35,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 37 msec 2023-07-16 21:15:35,669 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-16 21:15:35,697 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-16 21:15:35,703 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-16 21:15:35,711 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-07-16 21:15:35,717 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-16 21:15:35,722 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-07-16 21:15:35,726 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:15:35,727 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-07-16 21:15:35,727 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-07-16 21:15:35,741 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-07-16 21:15:35,746 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-16 21:15:35,746 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-16 21:15:35,746 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-16 21:15:35,747 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-16 21:15:35,747 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:15:35,750 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,41499,1689542132027, sessionid=0x1016ffd80780000, setting cluster-up flag (Was=false) 2023-07-16 21:15:35,768 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:15:35,773 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-07-16 21:15:35,775 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41499,1689542132027 2023-07-16 21:15:35,786 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:15:35,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-07-16 21:15:35,795 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41499,1689542132027 2023-07-16 21:15:35,798 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.hbase-snapshot/.tmp 2023-07-16 21:15:35,840 INFO [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer(951): ClusterId : bc8d5a8a-d305-489f-bc1d-b95b98296a6c 2023-07-16 21:15:35,841 INFO [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(951): ClusterId : bc8d5a8a-d305-489f-bc1d-b95b98296a6c 2023-07-16 21:15:35,840 INFO [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(951): ClusterId : bc8d5a8a-d305-489f-bc1d-b95b98296a6c 2023-07-16 21:15:35,847 DEBUG [RS:1;jenkins-hbase4:45229] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-16 21:15:35,847 DEBUG [RS:2;jenkins-hbase4:33511] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-16 21:15:35,847 DEBUG [RS:0;jenkins-hbase4:36993] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-16 21:15:35,854 DEBUG [RS:2;jenkins-hbase4:33511] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-16 21:15:35,854 DEBUG [RS:1;jenkins-hbase4:45229] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-16 21:15:35,854 DEBUG [RS:0;jenkins-hbase4:36993] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-16 21:15:35,854 DEBUG [RS:1;jenkins-hbase4:45229] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-16 21:15:35,854 DEBUG [RS:2;jenkins-hbase4:33511] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-16 21:15:35,854 DEBUG [RS:0;jenkins-hbase4:36993] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-16 21:15:35,859 DEBUG [RS:2;jenkins-hbase4:33511] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-16 21:15:35,859 DEBUG [RS:0;jenkins-hbase4:36993] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-16 21:15:35,859 DEBUG [RS:1;jenkins-hbase4:45229] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-16 21:15:35,860 DEBUG [RS:2;jenkins-hbase4:33511] zookeeper.ReadOnlyZKClient(139): Connect 0x783ea80a to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:15:35,861 DEBUG [RS:1;jenkins-hbase4:45229] zookeeper.ReadOnlyZKClient(139): Connect 0x6710a5a7 to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:15:35,861 DEBUG [RS:0;jenkins-hbase4:36993] zookeeper.ReadOnlyZKClient(139): Connect 0x285bc483 to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:15:35,867 DEBUG [RS:1;jenkins-hbase4:45229] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@e5f0591, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:15:35,867 DEBUG [RS:2;jenkins-hbase4:33511] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7e99a6ab, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:15:35,868 DEBUG [RS:1;jenkins-hbase4:45229] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@657ebc36, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:15:35,868 DEBUG [RS:2;jenkins-hbase4:33511] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@d15a37, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:15:35,869 DEBUG [RS:0;jenkins-hbase4:36993] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@78f30424, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:15:35,869 DEBUG [RS:0;jenkins-hbase4:36993] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@433508c5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:15:35,886 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2930): Registered master coprocessor service: service=RSGroupAdminService 2023-07-16 21:15:35,894 DEBUG [RS:2;jenkins-hbase4:33511] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:33511 2023-07-16 21:15:35,895 DEBUG [RS:0;jenkins-hbase4:36993] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:36993 2023-07-16 21:15:35,897 DEBUG [RS:1;jenkins-hbase4:45229] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:45229 2023-07-16 21:15:35,900 INFO [RS:2;jenkins-hbase4:33511] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-16 21:15:35,900 INFO [RS:1;jenkins-hbase4:45229] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-16 21:15:35,901 INFO [RS:1;jenkins-hbase4:45229] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-16 21:15:35,901 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] rsgroup.RSGroupInfoManagerImpl(537): Refreshing in Offline mode. 2023-07-16 21:15:35,900 INFO [RS:0;jenkins-hbase4:36993] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-16 21:15:35,901 INFO [RS:0;jenkins-hbase4:36993] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-16 21:15:35,901 DEBUG [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1022): About to register with Master. 2023-07-16 21:15:35,901 INFO [RS:2;jenkins-hbase4:33511] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-16 21:15:35,901 DEBUG [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1022): About to register with Master. 2023-07-16 21:15:35,901 DEBUG [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer(1022): About to register with Master. 2023-07-16 21:15:35,903 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:15:35,905 INFO [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41499,1689542132027 with isa=jenkins-hbase4.apache.org/172.31.14.131:33511, startcode=1689542134438 2023-07-16 21:15:35,905 INFO [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41499,1689542132027 with isa=jenkins-hbase4.apache.org/172.31.14.131:36993, startcode=1689542134044 2023-07-16 21:15:35,905 INFO [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41499,1689542132027 with isa=jenkins-hbase4.apache.org/172.31.14.131:45229, startcode=1689542134233 2023-07-16 21:15:35,906 INFO [master/jenkins-hbase4:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint loaded, priority=536870911. 2023-07-16 21:15:35,906 INFO [master/jenkins-hbase4:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver loaded, priority=536870912. 2023-07-16 21:15:35,960 DEBUG [RS:0;jenkins-hbase4:36993] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-16 21:15:35,961 DEBUG [RS:2;jenkins-hbase4:33511] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-16 21:15:35,960 DEBUG [RS:1;jenkins-hbase4:45229] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-16 21:15:36,041 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38001, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-16 21:15:36,041 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54749, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-16 21:15:36,041 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46585, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-16 21:15:36,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-16 21:15:36,059 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:36,072 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:36,072 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:36,107 DEBUG [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer(2830): Master is not running yet 2023-07-16 21:15:36,107 DEBUG [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(2830): Master is not running yet 2023-07-16 21:15:36,107 WARN [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-16 21:15:36,107 DEBUG [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(2830): Master is not running yet 2023-07-16 21:15:36,107 WARN [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-16 21:15:36,107 WARN [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-16 21:15:36,113 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-16 21:15:36,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-16 21:15:36,121 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-16 21:15:36,121 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-16 21:15:36,123 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-16 21:15:36,123 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-16 21:15:36,123 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-16 21:15:36,124 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-16 21:15:36,124 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-16 21:15:36,124 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,124 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:15:36,124 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,126 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689542166126 2023-07-16 21:15:36,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-16 21:15:36,134 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-16 21:15:36,135 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-16 21:15:36,136 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-16 21:15:36,139 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-16 21:15:36,144 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-16 21:15:36,145 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-16 21:15:36,145 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-16 21:15:36,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-16 21:15:36,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,148 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-16 21:15:36,151 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-16 21:15:36,151 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-16 21:15:36,155 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-16 21:15:36,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-16 21:15:36,162 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689542136158,5,FailOnTimeoutGroup] 2023-07-16 21:15:36,166 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689542136162,5,FailOnTimeoutGroup] 2023-07-16 21:15:36,167 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,167 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-16 21:15:36,168 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,168 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,208 INFO [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41499,1689542132027 with isa=jenkins-hbase4.apache.org/172.31.14.131:33511, startcode=1689542134438 2023-07-16 21:15:36,208 INFO [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41499,1689542132027 with isa=jenkins-hbase4.apache.org/172.31.14.131:45229, startcode=1689542134233 2023-07-16 21:15:36,208 INFO [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41499,1689542132027 with isa=jenkins-hbase4.apache.org/172.31.14.131:36993, startcode=1689542134044 2023-07-16 21:15:36,230 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41499] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:36,232 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:15:36,233 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 1 2023-07-16 21:15:36,234 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-16 21:15:36,235 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-16 21:15:36,236 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb 2023-07-16 21:15:36,238 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41499] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:36,239 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:15:36,239 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 2 2023-07-16 21:15:36,240 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41499] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:36,240 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:15:36,240 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 3 2023-07-16 21:15:36,240 DEBUG [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb 2023-07-16 21:15:36,241 DEBUG [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35261 2023-07-16 21:15:36,241 DEBUG [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=43223 2023-07-16 21:15:36,242 DEBUG [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb 2023-07-16 21:15:36,242 DEBUG [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35261 2023-07-16 21:15:36,242 DEBUG [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb 2023-07-16 21:15:36,242 DEBUG [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=43223 2023-07-16 21:15:36,242 DEBUG [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35261 2023-07-16 21:15:36,242 DEBUG [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=43223 2023-07-16 21:15:36,250 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:36,257 DEBUG [RS:0;jenkins-hbase4:36993] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:36,257 WARN [RS:0;jenkins-hbase4:36993] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:15:36,258 INFO [RS:0;jenkins-hbase4:36993] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:15:36,258 DEBUG [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:36,258 DEBUG [RS:1;jenkins-hbase4:45229] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:36,258 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45229,1689542134233] 2023-07-16 21:15:36,258 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33511,1689542134438] 2023-07-16 21:15:36,258 DEBUG [RS:2;jenkins-hbase4:33511] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:36,258 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36993,1689542134044] 2023-07-16 21:15:36,259 WARN [RS:2;jenkins-hbase4:33511] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:15:36,258 WARN [RS:1;jenkins-hbase4:45229] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:15:36,266 INFO [RS:1;jenkins-hbase4:45229] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:15:36,267 INFO [RS:2;jenkins-hbase4:33511] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:15:36,268 DEBUG [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:36,268 DEBUG [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:36,284 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:36,287 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-16 21:15:36,287 DEBUG [RS:2;jenkins-hbase4:33511] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:36,287 DEBUG [RS:0;jenkins-hbase4:36993] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:36,287 DEBUG [RS:1;jenkins-hbase4:45229] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:36,288 DEBUG [RS:2;jenkins-hbase4:33511] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:36,288 DEBUG [RS:0;jenkins-hbase4:36993] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:36,288 DEBUG [RS:1;jenkins-hbase4:45229] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:36,289 DEBUG [RS:2;jenkins-hbase4:33511] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:36,289 DEBUG [RS:1;jenkins-hbase4:45229] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:36,289 DEBUG [RS:0;jenkins-hbase4:36993] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:36,289 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info 2023-07-16 21:15:36,290 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-16 21:15:36,291 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:36,291 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-16 21:15:36,294 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/rep_barrier 2023-07-16 21:15:36,294 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-16 21:15:36,295 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:36,295 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-16 21:15:36,297 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table 2023-07-16 21:15:36,298 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-16 21:15:36,299 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:36,300 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740 2023-07-16 21:15:36,301 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740 2023-07-16 21:15:36,303 DEBUG [RS:1;jenkins-hbase4:45229] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-16 21:15:36,303 DEBUG [RS:2;jenkins-hbase4:33511] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-16 21:15:36,303 DEBUG [RS:0;jenkins-hbase4:36993] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-16 21:15:36,306 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-16 21:15:36,308 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-16 21:15:36,312 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:36,313 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11954267520, jitterRate=0.11332792043685913}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-16 21:15:36,313 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-16 21:15:36,313 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-16 21:15:36,313 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-16 21:15:36,313 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-16 21:15:36,313 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-16 21:15:36,313 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-16 21:15:36,314 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-16 21:15:36,314 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-16 21:15:36,316 INFO [RS:2;jenkins-hbase4:33511] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-16 21:15:36,317 INFO [RS:0;jenkins-hbase4:36993] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-16 21:15:36,316 INFO [RS:1;jenkins-hbase4:45229] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-16 21:15:36,322 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-16 21:15:36,322 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-16 21:15:36,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-16 21:15:36,343 INFO [RS:0;jenkins-hbase4:36993] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-16 21:15:36,343 INFO [RS:2;jenkins-hbase4:33511] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-16 21:15:36,343 INFO [RS:1;jenkins-hbase4:45229] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-16 21:15:36,353 INFO [RS:0;jenkins-hbase4:36993] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-16 21:15:36,354 INFO [RS:2;jenkins-hbase4:33511] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-16 21:15:36,353 INFO [RS:1;jenkins-hbase4:45229] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-16 21:15:36,355 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-16 21:15:36,354 INFO [RS:0;jenkins-hbase4:36993] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,354 INFO [RS:2;jenkins-hbase4:33511] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,355 INFO [RS:1;jenkins-hbase4:45229] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,356 INFO [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-16 21:15:36,356 INFO [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-16 21:15:36,356 INFO [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-16 21:15:36,360 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-16 21:15:36,366 INFO [RS:2;jenkins-hbase4:33511] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,366 INFO [RS:0;jenkins-hbase4:36993] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,366 INFO [RS:1;jenkins-hbase4:45229] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,367 DEBUG [RS:0;jenkins-hbase4:36993] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,366 DEBUG [RS:2;jenkins-hbase4:33511] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,367 DEBUG [RS:2;jenkins-hbase4:33511] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,367 DEBUG [RS:0;jenkins-hbase4:36993] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,367 DEBUG [RS:1;jenkins-hbase4:45229] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,368 DEBUG [RS:0;jenkins-hbase4:36993] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,368 DEBUG [RS:1;jenkins-hbase4:45229] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,367 DEBUG [RS:2;jenkins-hbase4:33511] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,368 DEBUG [RS:1;jenkins-hbase4:45229] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,368 DEBUG [RS:2;jenkins-hbase4:33511] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,368 DEBUG [RS:1;jenkins-hbase4:45229] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,368 DEBUG [RS:2;jenkins-hbase4:33511] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,368 DEBUG [RS:1;jenkins-hbase4:45229] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,368 DEBUG [RS:2;jenkins-hbase4:33511] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:15:36,368 DEBUG [RS:1;jenkins-hbase4:45229] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:15:36,368 DEBUG [RS:2;jenkins-hbase4:33511] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,368 DEBUG [RS:1;jenkins-hbase4:45229] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,368 DEBUG [RS:0;jenkins-hbase4:36993] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,368 DEBUG [RS:1;jenkins-hbase4:45229] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,369 DEBUG [RS:0;jenkins-hbase4:36993] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,369 DEBUG [RS:1;jenkins-hbase4:45229] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,369 DEBUG [RS:0;jenkins-hbase4:36993] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:15:36,369 DEBUG [RS:1;jenkins-hbase4:45229] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,369 DEBUG [RS:0;jenkins-hbase4:36993] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,368 DEBUG [RS:2;jenkins-hbase4:33511] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,369 DEBUG [RS:0;jenkins-hbase4:36993] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,369 DEBUG [RS:2;jenkins-hbase4:33511] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,370 DEBUG [RS:0;jenkins-hbase4:36993] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,370 DEBUG [RS:2;jenkins-hbase4:33511] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,370 DEBUG [RS:0;jenkins-hbase4:36993] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:36,381 INFO [RS:1;jenkins-hbase4:45229] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,381 INFO [RS:1;jenkins-hbase4:45229] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,381 INFO [RS:1;jenkins-hbase4:45229] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,386 INFO [RS:2;jenkins-hbase4:33511] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,387 INFO [RS:2;jenkins-hbase4:33511] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,387 INFO [RS:2;jenkins-hbase4:33511] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,399 INFO [RS:0;jenkins-hbase4:36993] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,399 INFO [RS:0;jenkins-hbase4:36993] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,399 INFO [RS:0;jenkins-hbase4:36993] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,403 INFO [RS:1;jenkins-hbase4:45229] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-16 21:15:36,406 INFO [RS:1;jenkins-hbase4:45229] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45229,1689542134233-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,414 INFO [RS:2;jenkins-hbase4:33511] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-16 21:15:36,415 INFO [RS:2;jenkins-hbase4:33511] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33511,1689542134438-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,422 INFO [RS:0;jenkins-hbase4:36993] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-16 21:15:36,422 INFO [RS:0;jenkins-hbase4:36993] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36993,1689542134044-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,439 INFO [RS:1;jenkins-hbase4:45229] regionserver.Replication(203): jenkins-hbase4.apache.org,45229,1689542134233 started 2023-07-16 21:15:36,439 INFO [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,45229,1689542134233, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45229, sessionid=0x1016ffd80780002 2023-07-16 21:15:36,439 INFO [RS:0;jenkins-hbase4:36993] regionserver.Replication(203): jenkins-hbase4.apache.org,36993,1689542134044 started 2023-07-16 21:15:36,439 DEBUG [RS:1;jenkins-hbase4:45229] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-16 21:15:36,439 INFO [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,36993,1689542134044, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36993, sessionid=0x1016ffd80780001 2023-07-16 21:15:36,440 DEBUG [RS:1;jenkins-hbase4:45229] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:36,440 DEBUG [RS:0;jenkins-hbase4:36993] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-16 21:15:36,440 DEBUG [RS:1;jenkins-hbase4:45229] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45229,1689542134233' 2023-07-16 21:15:36,440 DEBUG [RS:0;jenkins-hbase4:36993] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:36,443 DEBUG [RS:1;jenkins-hbase4:45229] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-16 21:15:36,443 DEBUG [RS:0;jenkins-hbase4:36993] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36993,1689542134044' 2023-07-16 21:15:36,444 DEBUG [RS:0;jenkins-hbase4:36993] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-16 21:15:36,445 DEBUG [RS:1;jenkins-hbase4:45229] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-16 21:15:36,445 DEBUG [RS:0;jenkins-hbase4:36993] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-16 21:15:36,445 DEBUG [RS:1;jenkins-hbase4:45229] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-16 21:15:36,445 DEBUG [RS:1;jenkins-hbase4:45229] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-16 21:15:36,446 DEBUG [RS:1;jenkins-hbase4:45229] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:36,446 DEBUG [RS:0;jenkins-hbase4:36993] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-16 21:15:36,446 DEBUG [RS:0;jenkins-hbase4:36993] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-16 21:15:36,446 DEBUG [RS:1;jenkins-hbase4:45229] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45229,1689542134233' 2023-07-16 21:15:36,446 DEBUG [RS:1;jenkins-hbase4:45229] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:36,446 DEBUG [RS:0;jenkins-hbase4:36993] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:36,447 DEBUG [RS:0;jenkins-hbase4:36993] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36993,1689542134044' 2023-07-16 21:15:36,447 DEBUG [RS:0;jenkins-hbase4:36993] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:36,448 INFO [RS:2;jenkins-hbase4:33511] regionserver.Replication(203): jenkins-hbase4.apache.org,33511,1689542134438 started 2023-07-16 21:15:36,448 INFO [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,33511,1689542134438, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33511, sessionid=0x1016ffd80780003 2023-07-16 21:15:36,448 DEBUG [RS:2;jenkins-hbase4:33511] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-16 21:15:36,448 DEBUG [RS:2;jenkins-hbase4:33511] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:36,448 DEBUG [RS:2;jenkins-hbase4:33511] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33511,1689542134438' 2023-07-16 21:15:36,448 DEBUG [RS:2;jenkins-hbase4:33511] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-16 21:15:36,449 DEBUG [RS:0;jenkins-hbase4:36993] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:36,449 DEBUG [RS:2;jenkins-hbase4:33511] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-16 21:15:36,450 DEBUG [RS:0;jenkins-hbase4:36993] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-16 21:15:36,450 DEBUG [RS:2;jenkins-hbase4:33511] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-16 21:15:36,450 INFO [RS:0;jenkins-hbase4:36993] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-16 21:15:36,450 DEBUG [RS:2;jenkins-hbase4:33511] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-16 21:15:36,450 INFO [RS:0;jenkins-hbase4:36993] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-16 21:15:36,450 DEBUG [RS:2;jenkins-hbase4:33511] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:36,450 DEBUG [RS:2;jenkins-hbase4:33511] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33511,1689542134438' 2023-07-16 21:15:36,450 DEBUG [RS:2;jenkins-hbase4:33511] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:36,450 DEBUG [RS:1;jenkins-hbase4:45229] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:36,451 DEBUG [RS:2;jenkins-hbase4:33511] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:36,451 DEBUG [RS:1;jenkins-hbase4:45229] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-16 21:15:36,451 INFO [RS:1;jenkins-hbase4:45229] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-16 21:15:36,452 INFO [RS:1;jenkins-hbase4:45229] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-16 21:15:36,452 DEBUG [RS:2;jenkins-hbase4:33511] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-16 21:15:36,452 INFO [RS:2;jenkins-hbase4:33511] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-16 21:15:36,452 INFO [RS:2;jenkins-hbase4:33511] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-16 21:15:36,512 DEBUG [jenkins-hbase4:41499] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-07-16 21:15:36,526 DEBUG [jenkins-hbase4:41499] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-16 21:15:36,529 DEBUG [jenkins-hbase4:41499] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-16 21:15:36,529 DEBUG [jenkins-hbase4:41499] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-16 21:15:36,529 DEBUG [jenkins-hbase4:41499] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-16 21:15:36,529 DEBUG [jenkins-hbase4:41499] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-16 21:15:36,532 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36993,1689542134044, state=OPENING 2023-07-16 21:15:36,540 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-16 21:15:36,542 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:15:36,542 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-16 21:15:36,546 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36993,1689542134044}] 2023-07-16 21:15:36,564 INFO [RS:1;jenkins-hbase4:45229] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45229%2C1689542134233, suffix=, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,45229,1689542134233, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs, maxLogs=32 2023-07-16 21:15:36,564 INFO [RS:2;jenkins-hbase4:33511] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33511%2C1689542134438, suffix=, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,33511,1689542134438, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs, maxLogs=32 2023-07-16 21:15:36,564 INFO [RS:0;jenkins-hbase4:36993] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36993%2C1689542134044, suffix=, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,36993,1689542134044, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs, maxLogs=32 2023-07-16 21:15:36,586 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:15:36,591 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:15:36,592 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:15:36,603 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:15:36,603 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:15:36,603 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:15:36,617 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:15:36,617 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:15:36,617 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:15:36,622 INFO [RS:2;jenkins-hbase4:33511] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,33511,1689542134438/jenkins-hbase4.apache.org%2C33511%2C1689542134438.1689542136569 2023-07-16 21:15:36,627 DEBUG [RS:2;jenkins-hbase4:33511] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK], DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK], DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK]] 2023-07-16 21:15:36,632 INFO [RS:1;jenkins-hbase4:45229] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,45229,1689542134233/jenkins-hbase4.apache.org%2C45229%2C1689542134233.1689542136569 2023-07-16 21:15:36,633 DEBUG [RS:1;jenkins-hbase4:45229] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK], DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK], DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK]] 2023-07-16 21:15:36,633 INFO [RS:0;jenkins-hbase4:36993] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,36993,1689542134044/jenkins-hbase4.apache.org%2C36993%2C1689542134044.1689542136569 2023-07-16 21:15:36,634 DEBUG [RS:0;jenkins-hbase4:36993] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK], DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK], DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK]] 2023-07-16 21:15:36,729 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:36,731 WARN [ReadOnlyZKClient-127.0.0.1:63917@0x084e601d] client.ZKConnectionRegistry(168): Meta region is in state OPENING 2023-07-16 21:15:36,732 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-16 21:15:36,735 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44098, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-16 21:15:36,748 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-16 21:15:36,748 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:15:36,752 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36993%2C1689542134044.meta, suffix=.meta, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,36993,1689542134044, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs, maxLogs=32 2023-07-16 21:15:36,766 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,41499,1689542132027] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:15:36,771 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44104, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:15:36,772 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36993] ipc.CallRunner(144): callId: 1 service: ClientService methodName: Get size: 88 connection: 172.31.14.131:44104 deadline: 1689542196771, exception=org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region hbase:meta,,1 is opening on jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:36,783 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:15:36,785 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:15:36,785 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:15:36,793 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,36993,1689542134044/jenkins-hbase4.apache.org%2C36993%2C1689542134044.meta.1689542136753.meta 2023-07-16 21:15:36,794 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK], DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK], DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK]] 2023-07-16 21:15:36,794 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:15:36,795 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-16 21:15:36,798 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-16 21:15:36,799 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-16 21:15:36,804 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-16 21:15:36,805 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:36,805 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-16 21:15:36,805 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-16 21:15:36,811 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-16 21:15:36,817 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info 2023-07-16 21:15:36,817 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info 2023-07-16 21:15:36,818 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-16 21:15:36,823 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:36,823 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-16 21:15:36,826 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/rep_barrier 2023-07-16 21:15:36,826 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/rep_barrier 2023-07-16 21:15:36,829 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-16 21:15:36,830 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:36,831 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-16 21:15:36,832 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table 2023-07-16 21:15:36,832 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table 2023-07-16 21:15:36,833 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-16 21:15:36,834 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:36,835 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740 2023-07-16 21:15:36,839 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740 2023-07-16 21:15:36,843 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-16 21:15:36,845 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-16 21:15:36,847 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10478149120, jitterRate=-0.024146318435668945}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-16 21:15:36,847 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-16 21:15:36,857 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689542136721 2023-07-16 21:15:36,878 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-16 21:15:36,879 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-16 21:15:36,879 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36993,1689542134044, state=OPEN 2023-07-16 21:15:36,882 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-16 21:15:36,882 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-16 21:15:36,886 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-16 21:15:36,886 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36993,1689542134044 in 336 msec 2023-07-16 21:15:36,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-16 21:15:36,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 555 msec 2023-07-16 21:15:36,898 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 982 msec 2023-07-16 21:15:36,898 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689542136898, completionTime=-1 2023-07-16 21:15:36,898 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-07-16 21:15:36,898 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-16 21:15:36,947 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-16 21:15:36,947 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689542196947 2023-07-16 21:15:36,947 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689542256947 2023-07-16 21:15:36,947 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 49 msec 2023-07-16 21:15:36,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41499,1689542132027-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,970 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41499,1689542132027-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,970 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41499,1689542132027-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,972 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:41499, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,972 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:36,979 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-16 21:15:36,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-16 21:15:36,992 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-16 21:15:37,002 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-16 21:15:37,004 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-16 21:15:37,008 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-16 21:15:37,026 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:37,029 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e empty. 2023-07-16 21:15:37,030 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:37,030 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-16 21:15:37,069 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-16 21:15:37,072 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 73c0efa65c224047ddc5ff1870c16b5e, NAME => 'hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:37,089 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:37,089 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 73c0efa65c224047ddc5ff1870c16b5e, disabling compactions & flushes 2023-07-16 21:15:37,090 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:15:37,090 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:15:37,090 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. after waiting 0 ms 2023-07-16 21:15:37,090 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:15:37,090 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:15:37,090 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 73c0efa65c224047ddc5ff1870c16b5e: 2023-07-16 21:15:37,094 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-16 21:15:37,111 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689542137097"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542137097"}]},"ts":"1689542137097"} 2023-07-16 21:15:37,141 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-16 21:15:37,143 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-16 21:15:37,148 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542137143"}]},"ts":"1689542137143"} 2023-07-16 21:15:37,152 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-16 21:15:37,156 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-16 21:15:37,156 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-16 21:15:37,156 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-16 21:15:37,156 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-16 21:15:37,156 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-16 21:15:37,158 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, ASSIGN}] 2023-07-16 21:15:37,161 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, ASSIGN 2023-07-16 21:15:37,162 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45229,1689542134233; forceNewPlan=false, retain=false 2023-07-16 21:15:37,297 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,41499,1689542132027] master.HMaster(2148): Client=null/null create 'hbase:rsgroup', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|', METADATA => {'SPLIT_POLICY' => 'org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy'}}}, {NAME => 'm', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-16 21:15:37,299 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,41499,1689542132027] procedure2.ProcedureExecutor(1029): Stored pid=6, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:rsgroup 2023-07-16 21:15:37,301 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=6, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:rsgroup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-16 21:15:37,303 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=6, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:rsgroup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-16 21:15:37,307 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:37,308 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db empty. 2023-07-16 21:15:37,309 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:37,309 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived hbase:rsgroup regions 2023-07-16 21:15:37,313 INFO [jenkins-hbase4:41499] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-16 21:15:37,315 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=73c0efa65c224047ddc5ff1870c16b5e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:37,315 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689542137314"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542137314"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542137314"}]},"ts":"1689542137314"} 2023-07-16 21:15:37,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=7, ppid=5, state=RUNNABLE; OpenRegionProcedure 73c0efa65c224047ddc5ff1870c16b5e, server=jenkins-hbase4.apache.org,45229,1689542134233}] 2023-07-16 21:15:37,352 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/hbase/rsgroup/.tabledesc/.tableinfo.0000000001 2023-07-16 21:15:37,354 INFO [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(7675): creating {ENCODED => 82d35ab01095f590b40a2bc17e2059db, NAME => 'hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:rsgroup', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|', METADATA => {'SPLIT_POLICY' => 'org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy'}}}, {NAME => 'm', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:37,386 DEBUG [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(866): Instantiated hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:37,386 DEBUG [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(1604): Closing 82d35ab01095f590b40a2bc17e2059db, disabling compactions & flushes 2023-07-16 21:15:37,386 INFO [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(1626): Closing region hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:37,386 DEBUG [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:37,387 DEBUG [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. after waiting 0 ms 2023-07-16 21:15:37,387 DEBUG [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:37,387 INFO [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(1838): Closed hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:37,387 DEBUG [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(1558): Region close journal for 82d35ab01095f590b40a2bc17e2059db: 2023-07-16 21:15:37,393 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=6, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:rsgroup execute state=CREATE_TABLE_ADD_TO_META 2023-07-16 21:15:37,395 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1689542137395"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542137395"}]},"ts":"1689542137395"} 2023-07-16 21:15:37,404 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-16 21:15:37,406 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=6, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:rsgroup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-16 21:15:37,406 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:rsgroup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542137406"}]},"ts":"1689542137406"} 2023-07-16 21:15:37,409 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=hbase:rsgroup, state=ENABLING in hbase:meta 2023-07-16 21:15:37,413 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-16 21:15:37,413 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-16 21:15:37,413 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-16 21:15:37,413 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-16 21:15:37,414 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-16 21:15:37,414 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=8, ppid=6, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, ASSIGN}] 2023-07-16 21:15:37,416 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=8, ppid=6, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, ASSIGN 2023-07-16 21:15:37,417 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=8, ppid=6, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45229,1689542134233; forceNewPlan=false, retain=false 2023-07-16 21:15:37,491 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:37,492 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-16 21:15:37,495 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58374, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-16 21:15:37,502 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:15:37,502 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 73c0efa65c224047ddc5ff1870c16b5e, NAME => 'hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:15:37,503 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:37,503 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:37,503 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:37,503 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:37,509 INFO [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:37,512 DEBUG [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/info 2023-07-16 21:15:37,512 DEBUG [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/info 2023-07-16 21:15:37,512 INFO [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 73c0efa65c224047ddc5ff1870c16b5e columnFamilyName info 2023-07-16 21:15:37,513 INFO [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] regionserver.HStore(310): Store=73c0efa65c224047ddc5ff1870c16b5e/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:37,514 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:37,515 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:37,519 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:37,527 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:37,528 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 73c0efa65c224047ddc5ff1870c16b5e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10502208800, jitterRate=-0.02190558612346649}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:37,528 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 73c0efa65c224047ddc5ff1870c16b5e: 2023-07-16 21:15:37,530 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e., pid=7, masterSystemTime=1689542137491 2023-07-16 21:15:37,535 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:15:37,535 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:15:37,537 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=73c0efa65c224047ddc5ff1870c16b5e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:37,537 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689542137536"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542137536"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542137536"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542137536"}]},"ts":"1689542137536"} 2023-07-16 21:15:37,544 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=7, resume processing ppid=5 2023-07-16 21:15:37,544 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=7, ppid=5, state=SUCCESS; OpenRegionProcedure 73c0efa65c224047ddc5ff1870c16b5e, server=jenkins-hbase4.apache.org,45229,1689542134233 in 218 msec 2023-07-16 21:15:37,547 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-16 21:15:37,548 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, ASSIGN in 386 msec 2023-07-16 21:15:37,549 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-16 21:15:37,549 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542137549"}]},"ts":"1689542137549"} 2023-07-16 21:15:37,553 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-16 21:15:37,557 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-16 21:15:37,561 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 565 msec 2023-07-16 21:15:37,568 INFO [jenkins-hbase4:41499] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-16 21:15:37,569 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=8 updating hbase:meta row=82d35ab01095f590b40a2bc17e2059db, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:37,569 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1689542137568"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542137568"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542137568"}]},"ts":"1689542137568"} 2023-07-16 21:15:37,571 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=9, ppid=8, state=RUNNABLE; OpenRegionProcedure 82d35ab01095f590b40a2bc17e2059db, server=jenkins-hbase4.apache.org,45229,1689542134233}] 2023-07-16 21:15:37,605 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-07-16 21:15:37,606 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-07-16 21:15:37,606 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:15:37,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:15:37,632 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58376, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:15:37,649 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=10, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-16 21:15:37,668 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-16 21:15:37,675 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=10, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 36 msec 2023-07-16 21:15:37,682 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=11, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-16 21:15:37,686 DEBUG [PEWorker-1] procedure.MasterProcedureScheduler(526): NAMESPACE 'hbase', shared lock count=1 2023-07-16 21:15:37,686 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=11, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-16 21:15:37,731 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:37,732 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 82d35ab01095f590b40a2bc17e2059db, NAME => 'hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:15:37,732 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-16 21:15:37,732 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. service=MultiRowMutationService 2023-07-16 21:15:37,734 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:rsgroup successfully. 2023-07-16 21:15:37,734 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table rsgroup 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:37,734 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:37,734 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:37,734 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:37,737 INFO [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family m of region 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:37,739 DEBUG [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m 2023-07-16 21:15:37,740 DEBUG [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m 2023-07-16 21:15:37,740 INFO [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 82d35ab01095f590b40a2bc17e2059db columnFamilyName m 2023-07-16 21:15:37,741 INFO [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] regionserver.HStore(310): Store=82d35ab01095f590b40a2bc17e2059db/m, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:37,743 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:37,745 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:37,751 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:37,755 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:37,756 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 82d35ab01095f590b40a2bc17e2059db; next sequenceid=2; org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy@35e06ed3, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:37,756 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 82d35ab01095f590b40a2bc17e2059db: 2023-07-16 21:15:37,758 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db., pid=9, masterSystemTime=1689542137724 2023-07-16 21:15:37,761 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:37,761 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:37,762 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=8 updating hbase:meta row=82d35ab01095f590b40a2bc17e2059db, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:37,762 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1689542137762"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542137762"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542137762"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542137762"}]},"ts":"1689542137762"} 2023-07-16 21:15:37,770 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=9, resume processing ppid=8 2023-07-16 21:15:37,770 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=9, ppid=8, state=SUCCESS; OpenRegionProcedure 82d35ab01095f590b40a2bc17e2059db, server=jenkins-hbase4.apache.org,45229,1689542134233 in 195 msec 2023-07-16 21:15:37,774 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=8, resume processing ppid=6 2023-07-16 21:15:37,776 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=8, ppid=6, state=SUCCESS; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, ASSIGN in 356 msec 2023-07-16 21:15:37,787 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-16 21:15:37,797 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=11, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 113 msec 2023-07-16 21:15:37,798 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=6, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:rsgroup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-16 21:15:37,799 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:rsgroup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542137798"}]},"ts":"1689542137798"} 2023-07-16 21:15:37,801 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=hbase:rsgroup, state=ENABLED in hbase:meta 2023-07-16 21:15:37,804 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=6, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:rsgroup execute state=CREATE_TABLE_POST_OPERATION 2023-07-16 21:15:37,807 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=6, state=SUCCESS; CreateTableProcedure table=hbase:rsgroup in 507 msec 2023-07-16 21:15:37,809 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-16 21:15:37,813 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-07-16 21:15:37,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 3.097sec 2023-07-16 21:15:37,815 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-16 21:15:37,817 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-16 21:15:37,817 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-16 21:15:37,818 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41499,1689542132027-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-16 21:15:37,819 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41499,1689542132027-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-16 21:15:37,834 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-16 21:15:37,855 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(139): Connect 0x7c41c5f7 to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:15:37,859 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3c8faac9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:15:37,875 DEBUG [hconnection-0x50aa0278-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:15:37,885 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44120, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:15:37,896 INFO [Listener at localhost/43605] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,41499,1689542132027 2023-07-16 21:15:37,897 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:37,906 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker(840): RSGroup table=hbase:rsgroup is online, refreshing cached information 2023-07-16 21:15:37,906 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl(534): Refreshing in Online mode. 2023-07-16 21:15:37,977 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:15:37,977 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:37,984 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 2 2023-07-16 21:15:37,990 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker(823): GroupBasedLoadBalancer is now online 2023-07-16 21:15:38,006 DEBUG [Listener at localhost/43605] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-16 21:15:38,012 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39404, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-16 21:15:38,024 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-07-16 21:15:38,024 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:15:38,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-16 21:15:38,031 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(139): Connect 0x2b6bba48 to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:15:38,037 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@defc5ad, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:15:38,037 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=VerifyingRSGroupAdminClient connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:15:38,040 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): VerifyingRSGroupAdminClient0x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:15:38,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): VerifyingRSGroupAdminClient-0x1016ffd8078000a connected 2023-07-16 21:15:38,077 INFO [Listener at localhost/43605] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testClearNotProcessedDeadServer Thread=424, OpenFileDescriptor=677, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=172, AvailableMemoryMB=3031 2023-07-16 21:15:38,079 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(132): testClearNotProcessedDeadServer 2023-07-16 21:15:38,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:38,107 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:38,154 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 1 2023-07-16 21:15:38,168 INFO [Listener at localhost/43605] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:15:38,168 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:38,168 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:38,168 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:15:38,168 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:38,168 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:15:38,168 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:15:38,172 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33295 2023-07-16 21:15:38,172 INFO [Listener at localhost/43605] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-16 21:15:38,173 DEBUG [Listener at localhost/43605] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-16 21:15:38,175 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:38,178 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:38,181 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33295 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:15:38,185 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:332950x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:15:38,187 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(162): regionserver:332950x0, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-16 21:15:38,187 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:33295-0x1016ffd8078000b connected 2023-07-16 21:15:38,188 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(162): regionserver:33295-0x1016ffd8078000b, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/running 2023-07-16 21:15:38,189 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:33295-0x1016ffd8078000b, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:15:38,190 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33295 2023-07-16 21:15:38,190 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33295 2023-07-16 21:15:38,190 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33295 2023-07-16 21:15:38,193 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33295 2023-07-16 21:15:38,193 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33295 2023-07-16 21:15:38,195 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:15:38,195 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:15:38,196 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:15:38,196 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-16 21:15:38,196 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:15:38,197 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:15:38,197 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:15:38,197 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 33373 2023-07-16 21:15:38,197 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:15:38,202 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:38,202 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@736a1f4f{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:15:38,203 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:38,203 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@4e1a3aab{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:15:38,327 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:15:38,328 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:15:38,329 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:15:38,329 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 600000ms 2023-07-16 21:15:38,330 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:38,331 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@2c08eb39{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-33373-hbase-server-2_4_18-SNAPSHOT_jar-_-any-5262144443991827390/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:15:38,332 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@2de685f1{HTTP/1.1, (http/1.1)}{0.0.0.0:33373} 2023-07-16 21:15:38,333 INFO [Listener at localhost/43605] server.Server(415): Started @12197ms 2023-07-16 21:15:38,337 INFO [RS:3;jenkins-hbase4:33295] regionserver.HRegionServer(951): ClusterId : bc8d5a8a-d305-489f-bc1d-b95b98296a6c 2023-07-16 21:15:38,340 DEBUG [RS:3;jenkins-hbase4:33295] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-16 21:15:38,346 DEBUG [RS:3;jenkins-hbase4:33295] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-16 21:15:38,347 DEBUG [RS:3;jenkins-hbase4:33295] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-16 21:15:38,349 DEBUG [RS:3;jenkins-hbase4:33295] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-16 21:15:38,350 DEBUG [RS:3;jenkins-hbase4:33295] zookeeper.ReadOnlyZKClient(139): Connect 0x4fc3fa71 to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:15:38,359 DEBUG [RS:3;jenkins-hbase4:33295] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@41040df9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:15:38,359 DEBUG [RS:3;jenkins-hbase4:33295] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@453b9101, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:15:38,371 DEBUG [RS:3;jenkins-hbase4:33295] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase4:33295 2023-07-16 21:15:38,371 INFO [RS:3;jenkins-hbase4:33295] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-16 21:15:38,371 INFO [RS:3;jenkins-hbase4:33295] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-16 21:15:38,371 DEBUG [RS:3;jenkins-hbase4:33295] regionserver.HRegionServer(1022): About to register with Master. 2023-07-16 21:15:38,373 INFO [RS:3;jenkins-hbase4:33295] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41499,1689542132027 with isa=jenkins-hbase4.apache.org/172.31.14.131:33295, startcode=1689542138167 2023-07-16 21:15:38,373 DEBUG [RS:3;jenkins-hbase4:33295] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-16 21:15:38,377 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58419, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-07-16 21:15:38,378 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41499] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33295,1689542138167 2023-07-16 21:15:38,379 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:15:38,383 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:38,384 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 2 2023-07-16 21:15:38,387 DEBUG [RS:3;jenkins-hbase4:33295] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb 2023-07-16 21:15:38,387 DEBUG [RS:3;jenkins-hbase4:33295] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35261 2023-07-16 21:15:38,387 DEBUG [RS:3;jenkins-hbase4:33295] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=43223 2023-07-16 21:15:38,398 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 4 2023-07-16 21:15:38,399 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:38,399 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:38,399 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:38,400 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:38,400 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33295,1689542138167] 2023-07-16 21:15:38,401 DEBUG [RS:3;jenkins-hbase4:33295] zookeeper.ZKUtil(162): regionserver:33295-0x1016ffd8078000b, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33295,1689542138167 2023-07-16 21:15:38,401 WARN [RS:3;jenkins-hbase4:33295] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:15:38,401 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:38,401 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:38,401 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:38,401 INFO [RS:3;jenkins-hbase4:33295] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:15:38,402 DEBUG [RS:3;jenkins-hbase4:33295] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,33295,1689542138167 2023-07-16 21:15:38,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33295,1689542138167 2023-07-16 21:15:38,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33295,1689542138167 2023-07-16 21:15:38,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33295,1689542138167 2023-07-16 21:15:38,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:38,404 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:38,404 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:38,404 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:38,404 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:38,404 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:38,418 DEBUG [RS:3;jenkins-hbase4:33295] zookeeper.ZKUtil(162): regionserver:33295-0x1016ffd8078000b, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:38,418 DEBUG [RS:3;jenkins-hbase4:33295] zookeeper.ZKUtil(162): regionserver:33295-0x1016ffd8078000b, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33295,1689542138167 2023-07-16 21:15:38,419 DEBUG [RS:3;jenkins-hbase4:33295] zookeeper.ZKUtil(162): regionserver:33295-0x1016ffd8078000b, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:38,419 DEBUG [RS:3;jenkins-hbase4:33295] zookeeper.ZKUtil(162): regionserver:33295-0x1016ffd8078000b, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:38,421 DEBUG [RS:3;jenkins-hbase4:33295] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-16 21:15:38,422 INFO [RS:3;jenkins-hbase4:33295] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-16 21:15:38,425 INFO [RS:3;jenkins-hbase4:33295] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-16 21:15:38,435 INFO [RS:3;jenkins-hbase4:33295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-16 21:15:38,435 INFO [RS:3;jenkins-hbase4:33295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:38,435 INFO [RS:3;jenkins-hbase4:33295] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-16 21:15:38,438 INFO [RS:3;jenkins-hbase4:33295] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:38,438 DEBUG [RS:3;jenkins-hbase4:33295] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,440 DEBUG [RS:3;jenkins-hbase4:33295] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,440 DEBUG [RS:3;jenkins-hbase4:33295] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,440 DEBUG [RS:3;jenkins-hbase4:33295] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,440 DEBUG [RS:3;jenkins-hbase4:33295] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,440 DEBUG [RS:3;jenkins-hbase4:33295] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:15:38,440 DEBUG [RS:3;jenkins-hbase4:33295] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,440 DEBUG [RS:3;jenkins-hbase4:33295] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,440 DEBUG [RS:3;jenkins-hbase4:33295] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,440 DEBUG [RS:3;jenkins-hbase4:33295] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,447 INFO [RS:3;jenkins-hbase4:33295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:38,448 INFO [RS:3;jenkins-hbase4:33295] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:38,448 INFO [RS:3;jenkins-hbase4:33295] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:38,461 INFO [RS:3;jenkins-hbase4:33295] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-16 21:15:38,461 INFO [RS:3;jenkins-hbase4:33295] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33295,1689542138167-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:38,473 INFO [RS:3;jenkins-hbase4:33295] regionserver.Replication(203): jenkins-hbase4.apache.org,33295,1689542138167 started 2023-07-16 21:15:38,473 INFO [RS:3;jenkins-hbase4:33295] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,33295,1689542138167, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33295, sessionid=0x1016ffd8078000b 2023-07-16 21:15:38,473 DEBUG [RS:3;jenkins-hbase4:33295] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-16 21:15:38,473 DEBUG [RS:3;jenkins-hbase4:33295] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33295,1689542138167 2023-07-16 21:15:38,473 DEBUG [RS:3;jenkins-hbase4:33295] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33295,1689542138167' 2023-07-16 21:15:38,473 DEBUG [RS:3;jenkins-hbase4:33295] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-16 21:15:38,474 DEBUG [RS:3;jenkins-hbase4:33295] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-16 21:15:38,474 DEBUG [RS:3;jenkins-hbase4:33295] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-16 21:15:38,474 DEBUG [RS:3;jenkins-hbase4:33295] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-16 21:15:38,474 DEBUG [RS:3;jenkins-hbase4:33295] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33295,1689542138167 2023-07-16 21:15:38,474 DEBUG [RS:3;jenkins-hbase4:33295] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33295,1689542138167' 2023-07-16 21:15:38,474 DEBUG [RS:3;jenkins-hbase4:33295] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:38,475 DEBUG [RS:3;jenkins-hbase4:33295] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:38,475 DEBUG [RS:3;jenkins-hbase4:33295] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-16 21:15:38,475 INFO [RS:3;jenkins-hbase4:33295] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-16 21:15:38,475 INFO [RS:3;jenkins-hbase4:33295] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-16 21:15:38,479 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:38,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:38,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:38,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:38,488 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:38,491 DEBUG [hconnection-0x609dbbf-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:15:38,494 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44134, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:15:38,498 DEBUG [hconnection-0x609dbbf-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:15:38,501 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58378, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:15:38,503 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:38,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:38,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:38,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:38,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] ipc.CallRunner(144): callId: 20 service: MasterService methodName: ExecMasterService size: 118 connection: 172.31.14.131:39404 deadline: 1689543338512, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:38,514 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:38,516 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:38,517 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:38,517 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:38,518 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33295, jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:38,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:38,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:38,524 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBasics(260): testClearNotProcessedDeadServer 2023-07-16 21:15:38,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:38,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:38,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup deadServerGroup 2023-07-16 21:15:38,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:38,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:38,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/deadServerGroup 2023-07-16 21:15:38,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:15:38,535 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:38,538 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:38,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:38,542 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:33295] to rsgroup deadServerGroup 2023-07-16 21:15:38,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:38,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:38,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/deadServerGroup 2023-07-16 21:15:38,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:15:38,549 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminServer(286): Moving 0 region(s) to group default, current retry=0 2023-07-16 21:15:38,549 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminServer(261): All regions from [jenkins-hbase4.apache.org,33295,1689542138167] are moved back to default 2023-07-16 21:15:38,549 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminServer(438): Move servers done: default => deadServerGroup 2023-07-16 21:15:38,549 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:38,553 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:38,553 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:38,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=deadServerGroup 2023-07-16 21:15:38,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:38,560 DEBUG [Listener at localhost/43605] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-16 21:15:38,564 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45038, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-16 21:15:38,564 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33295] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33295,1689542138167' ***** 2023-07-16 21:15:38,565 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33295] regionserver.HRegionServer(2311): STOPPED: Called by admin client hconnection-0x50aa0278 2023-07-16 21:15:38,565 INFO [RS:3;jenkins-hbase4:33295] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:15:38,571 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:38,576 INFO [RS:3;jenkins-hbase4:33295] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@2c08eb39{regionserver,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:15:38,581 INFO [RS:3;jenkins-hbase4:33295] server.AbstractConnector(383): Stopped ServerConnector@2de685f1{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:15:38,582 INFO [RS:3;jenkins-hbase4:33295] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:15:38,582 INFO [RS:3;jenkins-hbase4:33295] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@4e1a3aab{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:15:38,583 INFO [RS:3;jenkins-hbase4:33295] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@736a1f4f{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:15:38,585 INFO [RS:3;jenkins-hbase4:33295] regionserver.HeapMemoryManager(220): Stopping 2023-07-16 21:15:38,585 INFO [RS:3;jenkins-hbase4:33295] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-16 21:15:38,585 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-16 21:15:38,585 INFO [RS:3;jenkins-hbase4:33295] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-16 21:15:38,586 INFO [RS:3;jenkins-hbase4:33295] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33295,1689542138167 2023-07-16 21:15:38,586 DEBUG [RS:3;jenkins-hbase4:33295] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4fc3fa71 to 127.0.0.1:63917 2023-07-16 21:15:38,586 DEBUG [RS:3;jenkins-hbase4:33295] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:15:38,586 INFO [RS:3;jenkins-hbase4:33295] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33295,1689542138167; all regions closed. 2023-07-16 21:15:38,587 DEBUG [RS:3;jenkins-hbase4:33295] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:15:38,587 INFO [RS:3;jenkins-hbase4:33295] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:15:38,587 INFO [RS:3;jenkins-hbase4:33295] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-16 21:15:38,588 INFO [RS:3;jenkins-hbase4:33295] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-16 21:15:38,588 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:15:38,588 INFO [RS:3;jenkins-hbase4:33295] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-16 21:15:38,588 INFO [RS:3;jenkins-hbase4:33295] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-16 21:15:38,588 INFO [RS:3;jenkins-hbase4:33295] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33295 2023-07-16 21:15:38,601 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33295-0x1016ffd8078000b, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33295,1689542138167 2023-07-16 21:15:38,601 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33295,1689542138167 2023-07-16 21:15:38,601 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:38,602 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:38,602 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33295,1689542138167 2023-07-16 21:15:38,602 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:38,601 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33295-0x1016ffd8078000b, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:38,601 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33295,1689542138167 2023-07-16 21:15:38,603 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:38,604 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:38,605 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33295,1689542138167] 2023-07-16 21:15:38,605 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33295,1689542138167; numProcessing=1 2023-07-16 21:15:38,605 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:38,605 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:38,606 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:38,606 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:38,606 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,33295,1689542138167 already deleted, retry=false 2023-07-16 21:15:38,606 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:38,607 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,33295,1689542138167 znode expired, triggering replicatorRemoved event 2023-07-16 21:15:38,607 INFO [RegionServerTracker-0] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,33295,1689542138167 on jenkins-hbase4.apache.org,41499,1689542132027 2023-07-16 21:15:38,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:38,608 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,33295,1689542138167 znode expired, triggering replicatorRemoved event 2023-07-16 21:15:38,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:38,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:38,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:38,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:38,611 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,33295,1689542138167 znode expired, triggering replicatorRemoved event 2023-07-16 21:15:38,612 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:38,612 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:38,612 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:38,612 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:38,613 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:38,613 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:38,614 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:38,616 DEBUG [RegionServerTracker-0] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,33295,1689542138167, splitWal=true, meta=false 2023-07-16 21:15:38,616 INFO [RegionServerTracker-0] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=12 for jenkins-hbase4.apache.org,33295,1689542138167 (carryingMeta=false). 2023-07-16 21:15:38,617 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:15:38,620 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=12, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,33295,1689542138167, splitWal=true, meta=false 2023-07-16 21:15:38,621 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:38,622 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:38,622 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,33295,1689542138167 had 0 regions 2023-07-16 21:15:38,622 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/deadServerGroup 2023-07-16 21:15:38,623 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:15:38,623 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=12, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,33295,1689542138167, splitWal=true, meta=false, isMeta: false 2023-07-16 21:15:38,626 DEBUG [PEWorker-2] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,33295,1689542138167-splitting 2023-07-16 21:15:38,627 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 3 2023-07-16 21:15:38,628 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,33295,1689542138167-splitting dir is empty, no logs to split. 2023-07-16 21:15:38,628 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,33295,1689542138167 WAL count=0, meta=false 2023-07-16 21:15:38,632 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,33295,1689542138167-splitting dir is empty, no logs to split. 2023-07-16 21:15:38,632 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,33295,1689542138167 WAL count=0, meta=false 2023-07-16 21:15:38,632 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,33295,1689542138167 WAL splitting is done? wals=0, meta=false 2023-07-16 21:15:38,637 INFO [PEWorker-2] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,33295,1689542138167 failed, ignore...File hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,33295,1689542138167-splitting does not exist. 2023-07-16 21:15:38,639 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,33295,1689542138167 after splitting done 2023-07-16 21:15:38,640 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,33295,1689542138167 from processing; numProcessing=0 2023-07-16 21:15:38,642 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,33295,1689542138167, splitWal=true, meta=false in 31 msec 2023-07-16 21:15:38,648 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:15:38,674 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=deadServerGroup 2023-07-16 21:15:38,674 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:38,683 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:38,683 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:38,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:38,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:38,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:38,695 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:38,695 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:38,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:38,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:38,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/deadServerGroup 2023-07-16 21:15:38,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 5 2023-07-16 21:15:38,718 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:38,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:38,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:38,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:38,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:33295] to rsgroup default 2023-07-16 21:15:38,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(258): Dropping jenkins-hbase4.apache.org:33295 during move-to-default rsgroup because not online 2023-07-16 21:15:38,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:38,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/deadServerGroup 2023-07-16 21:15:38,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:38,732 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminServer(286): Moving 0 region(s) to group deadServerGroup, current retry=0 2023-07-16 21:15:38,732 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminServer(261): All regions from [] are moved back to deadServerGroup 2023-07-16 21:15:38,732 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminServer(438): Move servers done: deadServerGroup => default 2023-07-16 21:15:38,732 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:38,734 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup deadServerGroup 2023-07-16 21:15:38,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:38,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:38,745 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:38,751 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 1 2023-07-16 21:15:38,767 INFO [Listener at localhost/43605] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:15:38,768 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:38,768 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:38,768 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:15:38,768 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:38,768 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:15:38,768 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:15:38,771 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41213 2023-07-16 21:15:38,772 INFO [Listener at localhost/43605] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-16 21:15:38,773 DEBUG [Listener at localhost/43605] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-16 21:15:38,774 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:38,775 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:38,777 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41213 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:15:38,780 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:412130x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:15:38,782 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41213-0x1016ffd8078000d connected 2023-07-16 21:15:38,782 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(162): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-16 21:15:38,783 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(162): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/running 2023-07-16 21:15:38,784 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:15:38,787 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41213 2023-07-16 21:15:38,790 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41213 2023-07-16 21:15:38,790 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33295-0x1016ffd8078000b, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:15:38,790 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33295-0x1016ffd8078000b, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:15:38,791 INFO [RS:3;jenkins-hbase4:33295] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33295,1689542138167; zookeeper connection closed. 2023-07-16 21:15:38,792 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41213 2023-07-16 21:15:38,792 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@731d0216] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@731d0216 2023-07-16 21:15:38,798 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41213 2023-07-16 21:15:38,799 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41213 2023-07-16 21:15:38,801 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:15:38,802 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:15:38,802 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:15:38,802 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-16 21:15:38,803 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:15:38,803 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:15:38,803 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:15:38,804 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 45737 2023-07-16 21:15:38,804 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:15:38,807 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:38,807 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@78bafdff{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:15:38,807 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:38,808 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@67ca648{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:15:38,928 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:15:38,929 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:15:38,929 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:15:38,929 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 600000ms 2023-07-16 21:15:38,930 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:38,931 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@1899cb6d{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-45737-hbase-server-2_4_18-SNAPSHOT_jar-_-any-3312773915895731660/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:15:38,933 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@9b30d54{HTTP/1.1, (http/1.1)}{0.0.0.0:45737} 2023-07-16 21:15:38,933 INFO [Listener at localhost/43605] server.Server(415): Started @12797ms 2023-07-16 21:15:38,935 INFO [RS:4;jenkins-hbase4:41213] regionserver.HRegionServer(951): ClusterId : bc8d5a8a-d305-489f-bc1d-b95b98296a6c 2023-07-16 21:15:38,936 DEBUG [RS:4;jenkins-hbase4:41213] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-16 21:15:38,938 DEBUG [RS:4;jenkins-hbase4:41213] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-16 21:15:38,938 DEBUG [RS:4;jenkins-hbase4:41213] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-16 21:15:38,940 DEBUG [RS:4;jenkins-hbase4:41213] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-16 21:15:38,941 DEBUG [RS:4;jenkins-hbase4:41213] zookeeper.ReadOnlyZKClient(139): Connect 0x1f4f67ca to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:15:38,947 DEBUG [RS:4;jenkins-hbase4:41213] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@11888a53, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:15:38,947 DEBUG [RS:4;jenkins-hbase4:41213] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@15b70cba, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:15:38,956 DEBUG [RS:4;jenkins-hbase4:41213] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:4;jenkins-hbase4:41213 2023-07-16 21:15:38,956 INFO [RS:4;jenkins-hbase4:41213] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-16 21:15:38,956 INFO [RS:4;jenkins-hbase4:41213] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-16 21:15:38,956 DEBUG [RS:4;jenkins-hbase4:41213] regionserver.HRegionServer(1022): About to register with Master. 2023-07-16 21:15:38,957 INFO [RS:4;jenkins-hbase4:41213] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41499,1689542132027 with isa=jenkins-hbase4.apache.org/172.31.14.131:41213, startcode=1689542138767 2023-07-16 21:15:38,957 DEBUG [RS:4;jenkins-hbase4:41213] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-16 21:15:38,959 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54865, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-07-16 21:15:38,960 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41499] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:38,960 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:15:38,960 DEBUG [RS:4;jenkins-hbase4:41213] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb 2023-07-16 21:15:38,960 DEBUG [RS:4;jenkins-hbase4:41213] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35261 2023-07-16 21:15:38,960 DEBUG [RS:4;jenkins-hbase4:41213] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=43223 2023-07-16 21:15:38,962 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:38,962 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:38,962 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:38,962 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:38,963 DEBUG [RS:4;jenkins-hbase4:41213] zookeeper.ZKUtil(162): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:38,963 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41213,1689542138767] 2023-07-16 21:15:38,963 WARN [RS:4;jenkins-hbase4:41213] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:15:38,963 INFO [RS:4;jenkins-hbase4:41213] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:15:38,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:38,963 DEBUG [RS:4;jenkins-hbase4:41213] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:38,964 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:38,964 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:38,964 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:38,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:38,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:38,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:38,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:38,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:38,968 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:38,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:38,969 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 2 2023-07-16 21:15:38,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:38,971 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,41499,1689542132027] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 4 2023-07-16 21:15:38,971 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:38,972 DEBUG [RS:4;jenkins-hbase4:41213] zookeeper.ZKUtil(162): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:38,972 DEBUG [RS:4;jenkins-hbase4:41213] zookeeper.ZKUtil(162): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:38,973 DEBUG [RS:4;jenkins-hbase4:41213] zookeeper.ZKUtil(162): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:38,973 DEBUG [RS:4;jenkins-hbase4:41213] zookeeper.ZKUtil(162): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:38,974 DEBUG [RS:4;jenkins-hbase4:41213] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-16 21:15:38,975 INFO [RS:4;jenkins-hbase4:41213] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-16 21:15:38,977 INFO [RS:4;jenkins-hbase4:41213] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-16 21:15:38,978 INFO [RS:4;jenkins-hbase4:41213] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-16 21:15:38,978 INFO [RS:4;jenkins-hbase4:41213] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:38,985 INFO [RS:4;jenkins-hbase4:41213] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-16 21:15:38,987 INFO [RS:4;jenkins-hbase4:41213] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:38,988 DEBUG [RS:4;jenkins-hbase4:41213] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,988 DEBUG [RS:4;jenkins-hbase4:41213] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,988 DEBUG [RS:4;jenkins-hbase4:41213] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,988 DEBUG [RS:4;jenkins-hbase4:41213] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,988 DEBUG [RS:4;jenkins-hbase4:41213] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,988 DEBUG [RS:4;jenkins-hbase4:41213] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:15:38,988 DEBUG [RS:4;jenkins-hbase4:41213] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,988 DEBUG [RS:4;jenkins-hbase4:41213] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,988 DEBUG [RS:4;jenkins-hbase4:41213] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,988 DEBUG [RS:4;jenkins-hbase4:41213] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:38,991 INFO [RS:4;jenkins-hbase4:41213] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:38,991 INFO [RS:4;jenkins-hbase4:41213] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:38,991 INFO [RS:4;jenkins-hbase4:41213] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:39,003 INFO [RS:4;jenkins-hbase4:41213] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-16 21:15:39,003 INFO [RS:4;jenkins-hbase4:41213] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41213,1689542138767-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:39,013 INFO [RS:4;jenkins-hbase4:41213] regionserver.Replication(203): jenkins-hbase4.apache.org,41213,1689542138767 started 2023-07-16 21:15:39,014 INFO [RS:4;jenkins-hbase4:41213] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,41213,1689542138767, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41213, sessionid=0x1016ffd8078000d 2023-07-16 21:15:39,014 DEBUG [RS:4;jenkins-hbase4:41213] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-16 21:15:39,014 DEBUG [RS:4;jenkins-hbase4:41213] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:39,014 DEBUG [RS:4;jenkins-hbase4:41213] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41213,1689542138767' 2023-07-16 21:15:39,014 DEBUG [RS:4;jenkins-hbase4:41213] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-16 21:15:39,014 DEBUG [RS:4;jenkins-hbase4:41213] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-16 21:15:39,015 DEBUG [RS:4;jenkins-hbase4:41213] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-16 21:15:39,015 DEBUG [RS:4;jenkins-hbase4:41213] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-16 21:15:39,015 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:39,015 DEBUG [RS:4;jenkins-hbase4:41213] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:39,015 DEBUG [RS:4;jenkins-hbase4:41213] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41213,1689542138767' 2023-07-16 21:15:39,015 DEBUG [RS:4;jenkins-hbase4:41213] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:39,016 DEBUG [RS:4;jenkins-hbase4:41213] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:39,016 DEBUG [RS:4;jenkins-hbase4:41213] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-16 21:15:39,016 INFO [RS:4;jenkins-hbase4:41213] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-16 21:15:39,016 INFO [RS:4;jenkins-hbase4:41213] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-16 21:15:39,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:39,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:39,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:39,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:39,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:39,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:39,031 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:39,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:39,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] ipc.CallRunner(144): callId: 69 service: MasterService methodName: ExecMasterService size: 118 connection: 172.31.14.131:39404 deadline: 1689543339030, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:39,032 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:39,033 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:39,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:39,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:39,035 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:39,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:39,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:39,058 INFO [Listener at localhost/43605] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testClearNotProcessedDeadServer Thread=478 (was 424) Potentially hanging thread: RS:4;jenkins-hbase4:41213 java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1092) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:175) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:123) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:159) java.security.AccessController.doPrivileged(Native Method) javax.security.auth.Subject.doAs(Subject.java:360) org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1873) org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:319) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:156) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp254524763-699 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (409621679) connection to localhost/127.0.0.1:35261 from jenkins.hfs.3 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: qtp254524763-701 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:4;jenkins-hbase4:41213-longCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=41213 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp254524763-704 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:302) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:366) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41213 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41213 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: hconnection-0x68365814-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp254524763-700 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=41213 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x1f4f67ca-SendThread(127.0.0.1:63917) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: hconnection-0x609dbbf-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=41213 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp254524763-703 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x1f4f67ca sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$93/298992337.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41213 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41213 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41213 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (409621679) connection to localhost/127.0.0.1:35261 from jenkins.hfs.4 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: qtp254524763-697 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/234338121.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: jenkins-hbase4:41213Replication Statistics #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Session-HouseKeeper-26144534-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41213 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41213 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp254524763-702 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp254524763-698-acceptor-0@54bee7e9-ServerConnector@9b30d54{HTTP/1.1, (http/1.1)}{0.0.0.0:45737} sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.hbase.thirdparty.org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) org.apache.hbase.thirdparty.org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:704) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x1f4f67ca-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) - Thread LEAK? -, OpenFileDescriptor=719 (was 677) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 339), ProcessCount=172 (was 172), AvailableMemoryMB=3018 (was 3031) 2023-07-16 21:15:39,075 INFO [Listener at localhost/43605] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testDefaultNamespaceCreateAndAssign Thread=478, OpenFileDescriptor=719, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=172, AvailableMemoryMB=3017 2023-07-16 21:15:39,076 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(132): testDefaultNamespaceCreateAndAssign 2023-07-16 21:15:39,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:39,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:39,085 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:39,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:39,085 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:39,086 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:39,086 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:39,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:39,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:39,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:39,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:39,102 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:15:39,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:39,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:39,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:39,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:39,111 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:39,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:39,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:39,120 INFO [RS:4;jenkins-hbase4:41213] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41213%2C1689542138767, suffix=, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,41213,1689542138767, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs, maxLogs=32 2023-07-16 21:15:39,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:39,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:39,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] ipc.CallRunner(144): callId: 97 service: MasterService methodName: ExecMasterService size: 118 connection: 172.31.14.131:39404 deadline: 1689543339120, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:39,132 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:39,136 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:39,138 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:39,138 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:39,139 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:39,140 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:39,140 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:39,141 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBasics(180): testDefaultNamespaceCreateAndAssign 2023-07-16 21:15:39,150 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$16(3053): Client=jenkins//172.31.14.131 modify {NAME => 'default', hbase.rsgroup.name => 'default'} 2023-07-16 21:15:39,165 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:15:39,165 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:15:39,165 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:15:39,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_NAMESPACE_PREPARE; ModifyNamespaceProcedure, namespace=default 2023-07-16 21:15:39,183 INFO [RS:4;jenkins-hbase4:41213] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,41213,1689542138767/jenkins-hbase4.apache.org%2C41213%2C1689542138767.1689542139121 2023-07-16 21:15:39,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=13 2023-07-16 21:15:39,186 DEBUG [RS:4;jenkins-hbase4:41213] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK], DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK], DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK]] 2023-07-16 21:15:39,188 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-16 21:15:39,191 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyNamespaceProcedure, namespace=default in 38 msec 2023-07-16 21:15:39,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=13 2023-07-16 21:15:39,299 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'Group_testCreateAndAssign', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-16 21:15:39,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=14, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=Group_testCreateAndAssign 2023-07-16 21:15:39,302 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=14, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=Group_testCreateAndAssign execute state=CREATE_TABLE_PRE_OPERATION 2023-07-16 21:15:39,307 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "Group_testCreateAndAssign" procId is: 14 2023-07-16 21:15:39,308 DEBUG [PEWorker-1] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:39,308 DEBUG [PEWorker-1] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:39,309 DEBUG [PEWorker-1] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:39,311 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=14, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=Group_testCreateAndAssign execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-16 21:15:39,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=14 2023-07-16 21:15:39,314 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndAssign/e39fdb9ab9ff035bebdf3f7d19e4fb20 2023-07-16 21:15:39,314 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndAssign/e39fdb9ab9ff035bebdf3f7d19e4fb20 empty. 2023-07-16 21:15:39,315 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndAssign/e39fdb9ab9ff035bebdf3f7d19e4fb20 2023-07-16 21:15:39,315 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived Group_testCreateAndAssign regions 2023-07-16 21:15:39,342 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndAssign/.tabledesc/.tableinfo.0000000001 2023-07-16 21:15:39,343 INFO [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(7675): creating {ENCODED => e39fdb9ab9ff035bebdf3f7d19e4fb20, NAME => 'Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='Group_testCreateAndAssign', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:39,361 DEBUG [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(866): Instantiated Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:39,362 DEBUG [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(1604): Closing e39fdb9ab9ff035bebdf3f7d19e4fb20, disabling compactions & flushes 2023-07-16 21:15:39,362 INFO [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(1626): Closing region Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20. 2023-07-16 21:15:39,362 DEBUG [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20. 2023-07-16 21:15:39,362 DEBUG [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20. after waiting 0 ms 2023-07-16 21:15:39,362 DEBUG [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20. 2023-07-16 21:15:39,362 INFO [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(1838): Closed Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20. 2023-07-16 21:15:39,362 DEBUG [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(1558): Region close journal for e39fdb9ab9ff035bebdf3f7d19e4fb20: 2023-07-16 21:15:39,366 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=14, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=Group_testCreateAndAssign execute state=CREATE_TABLE_ADD_TO_META 2023-07-16 21:15:39,368 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1689542139367"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542139367"}]},"ts":"1689542139367"} 2023-07-16 21:15:39,370 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-16 21:15:39,371 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=14, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=Group_testCreateAndAssign execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-16 21:15:39,372 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542139371"}]},"ts":"1689542139371"} 2023-07-16 21:15:39,374 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndAssign, state=ENABLING in hbase:meta 2023-07-16 21:15:39,377 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-16 21:15:39,378 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-16 21:15:39,378 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-16 21:15:39,378 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-16 21:15:39,378 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-16 21:15:39,378 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-16 21:15:39,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateAndAssign, region=e39fdb9ab9ff035bebdf3f7d19e4fb20, ASSIGN}] 2023-07-16 21:15:39,380 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateAndAssign, region=e39fdb9ab9ff035bebdf3f7d19e4fb20, ASSIGN 2023-07-16 21:15:39,381 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateAndAssign, region=e39fdb9ab9ff035bebdf3f7d19e4fb20, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45229,1689542134233; forceNewPlan=false, retain=false 2023-07-16 21:15:39,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=14 2023-07-16 21:15:39,532 INFO [jenkins-hbase4:41499] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-16 21:15:39,533 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=e39fdb9ab9ff035bebdf3f7d19e4fb20, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:39,533 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1689542139533"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542139533"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542139533"}]},"ts":"1689542139533"} 2023-07-16 21:15:39,536 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; OpenRegionProcedure e39fdb9ab9ff035bebdf3f7d19e4fb20, server=jenkins-hbase4.apache.org,45229,1689542134233}] 2023-07-16 21:15:39,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=14 2023-07-16 21:15:39,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20. 2023-07-16 21:15:39,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e39fdb9ab9ff035bebdf3f7d19e4fb20, NAME => 'Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:15:39,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateAndAssign e39fdb9ab9ff035bebdf3f7d19e4fb20 2023-07-16 21:15:39,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:39,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e39fdb9ab9ff035bebdf3f7d19e4fb20 2023-07-16 21:15:39,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e39fdb9ab9ff035bebdf3f7d19e4fb20 2023-07-16 21:15:39,696 INFO [StoreOpener-e39fdb9ab9ff035bebdf3f7d19e4fb20-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region e39fdb9ab9ff035bebdf3f7d19e4fb20 2023-07-16 21:15:39,698 DEBUG [StoreOpener-e39fdb9ab9ff035bebdf3f7d19e4fb20-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateAndAssign/e39fdb9ab9ff035bebdf3f7d19e4fb20/f 2023-07-16 21:15:39,698 DEBUG [StoreOpener-e39fdb9ab9ff035bebdf3f7d19e4fb20-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateAndAssign/e39fdb9ab9ff035bebdf3f7d19e4fb20/f 2023-07-16 21:15:39,699 INFO [StoreOpener-e39fdb9ab9ff035bebdf3f7d19e4fb20-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e39fdb9ab9ff035bebdf3f7d19e4fb20 columnFamilyName f 2023-07-16 21:15:39,700 INFO [StoreOpener-e39fdb9ab9ff035bebdf3f7d19e4fb20-1] regionserver.HStore(310): Store=e39fdb9ab9ff035bebdf3f7d19e4fb20/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:39,701 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateAndAssign/e39fdb9ab9ff035bebdf3f7d19e4fb20 2023-07-16 21:15:39,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateAndAssign/e39fdb9ab9ff035bebdf3f7d19e4fb20 2023-07-16 21:15:39,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e39fdb9ab9ff035bebdf3f7d19e4fb20 2023-07-16 21:15:39,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateAndAssign/e39fdb9ab9ff035bebdf3f7d19e4fb20/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:39,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e39fdb9ab9ff035bebdf3f7d19e4fb20; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10832224800, jitterRate=0.008829548954963684}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:39,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e39fdb9ab9ff035bebdf3f7d19e4fb20: 2023-07-16 21:15:39,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20., pid=16, masterSystemTime=1689542139688 2023-07-16 21:15:39,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20. 2023-07-16 21:15:39,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20. 2023-07-16 21:15:39,713 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=e39fdb9ab9ff035bebdf3f7d19e4fb20, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:39,714 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1689542139713"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542139713"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542139713"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542139713"}]},"ts":"1689542139713"} 2023-07-16 21:15:39,719 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-07-16 21:15:39,719 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; OpenRegionProcedure e39fdb9ab9ff035bebdf3f7d19e4fb20, server=jenkins-hbase4.apache.org,45229,1689542134233 in 180 msec 2023-07-16 21:15:39,722 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-07-16 21:15:39,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateAndAssign, region=e39fdb9ab9ff035bebdf3f7d19e4fb20, ASSIGN in 341 msec 2023-07-16 21:15:39,724 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=14, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=Group_testCreateAndAssign execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-16 21:15:39,724 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542139724"}]},"ts":"1689542139724"} 2023-07-16 21:15:39,726 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndAssign, state=ENABLED in hbase:meta 2023-07-16 21:15:39,728 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=14, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=Group_testCreateAndAssign execute state=CREATE_TABLE_POST_OPERATION 2023-07-16 21:15:39,730 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, state=SUCCESS; CreateTableProcedure table=Group_testCreateAndAssign in 430 msec 2023-07-16 21:15:39,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=14 2023-07-16 21:15:39,916 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:Group_testCreateAndAssign, procId: 14 completed 2023-07-16 21:15:39,917 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:39,923 DEBUG [Listener at localhost/43605] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-16 21:15:39,927 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45756, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-16 21:15:39,930 DEBUG [Listener at localhost/43605] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-16 21:15:39,933 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44140, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-16 21:15:39,935 DEBUG [Listener at localhost/43605] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-16 21:15:39,937 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55100, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-16 21:15:39,938 DEBUG [Listener at localhost/43605] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-16 21:15:39,940 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58392, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-16 21:15:39,944 INFO [Listener at localhost/43605] client.HBaseAdmin$15(890): Started disable of Group_testCreateAndAssign 2023-07-16 21:15:39,948 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable Group_testCreateAndAssign 2023-07-16 21:15:39,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=17, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=Group_testCreateAndAssign 2023-07-16 21:15:39,959 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542139959"}]},"ts":"1689542139959"} 2023-07-16 21:15:39,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=17 2023-07-16 21:15:39,961 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndAssign, state=DISABLING in hbase:meta 2023-07-16 21:15:39,968 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set Group_testCreateAndAssign to state=DISABLING 2023-07-16 21:15:39,970 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateAndAssign, region=e39fdb9ab9ff035bebdf3f7d19e4fb20, UNASSIGN}] 2023-07-16 21:15:39,972 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=18, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateAndAssign, region=e39fdb9ab9ff035bebdf3f7d19e4fb20, UNASSIGN 2023-07-16 21:15:39,973 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=18 updating hbase:meta row=e39fdb9ab9ff035bebdf3f7d19e4fb20, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:39,973 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1689542139973"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542139973"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542139973"}]},"ts":"1689542139973"} 2023-07-16 21:15:39,975 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=18, state=RUNNABLE; CloseRegionProcedure e39fdb9ab9ff035bebdf3f7d19e4fb20, server=jenkins-hbase4.apache.org,45229,1689542134233}] 2023-07-16 21:15:40,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=17 2023-07-16 21:15:40,135 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close e39fdb9ab9ff035bebdf3f7d19e4fb20 2023-07-16 21:15:40,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e39fdb9ab9ff035bebdf3f7d19e4fb20, disabling compactions & flushes 2023-07-16 21:15:40,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20. 2023-07-16 21:15:40,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20. 2023-07-16 21:15:40,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20. after waiting 0 ms 2023-07-16 21:15:40,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20. 2023-07-16 21:15:40,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateAndAssign/e39fdb9ab9ff035bebdf3f7d19e4fb20/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:15:40,144 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20. 2023-07-16 21:15:40,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e39fdb9ab9ff035bebdf3f7d19e4fb20: 2023-07-16 21:15:40,146 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed e39fdb9ab9ff035bebdf3f7d19e4fb20 2023-07-16 21:15:40,147 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=18 updating hbase:meta row=e39fdb9ab9ff035bebdf3f7d19e4fb20, regionState=CLOSED 2023-07-16 21:15:40,148 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1689542140147"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542140147"}]},"ts":"1689542140147"} 2023-07-16 21:15:40,152 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=18 2023-07-16 21:15:40,152 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=18, state=SUCCESS; CloseRegionProcedure e39fdb9ab9ff035bebdf3f7d19e4fb20, server=jenkins-hbase4.apache.org,45229,1689542134233 in 175 msec 2023-07-16 21:15:40,155 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=17 2023-07-16 21:15:40,155 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=17, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateAndAssign, region=e39fdb9ab9ff035bebdf3f7d19e4fb20, UNASSIGN in 182 msec 2023-07-16 21:15:40,156 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542140156"}]},"ts":"1689542140156"} 2023-07-16 21:15:40,158 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndAssign, state=DISABLED in hbase:meta 2023-07-16 21:15:40,159 INFO [PEWorker-5] procedure.DisableTableProcedure(305): Set Group_testCreateAndAssign to state=DISABLED 2023-07-16 21:15:40,162 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=17, state=SUCCESS; DisableTableProcedure table=Group_testCreateAndAssign in 210 msec 2023-07-16 21:15:40,263 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=17 2023-07-16 21:15:40,264 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:Group_testCreateAndAssign, procId: 17 completed 2023-07-16 21:15:40,270 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete Group_testCreateAndAssign 2023-07-16 21:15:40,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=20, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-16 21:15:40,279 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=20, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-16 21:15:40,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint(577): Removing deleted table 'Group_testCreateAndAssign' from rsgroup 'default' 2023-07-16 21:15:40,281 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=20, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-16 21:15:40,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:40,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:40,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:40,289 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndAssign/e39fdb9ab9ff035bebdf3f7d19e4fb20 2023-07-16 21:15:40,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=20 2023-07-16 21:15:40,293 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndAssign/e39fdb9ab9ff035bebdf3f7d19e4fb20/f, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndAssign/e39fdb9ab9ff035bebdf3f7d19e4fb20/recovered.edits] 2023-07-16 21:15:40,307 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndAssign/e39fdb9ab9ff035bebdf3f7d19e4fb20/recovered.edits/4.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/default/Group_testCreateAndAssign/e39fdb9ab9ff035bebdf3f7d19e4fb20/recovered.edits/4.seqid 2023-07-16 21:15:40,311 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndAssign/e39fdb9ab9ff035bebdf3f7d19e4fb20 2023-07-16 21:15:40,312 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived Group_testCreateAndAssign regions 2023-07-16 21:15:40,316 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=20, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-16 21:15:40,339 WARN [PEWorker-2] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of Group_testCreateAndAssign from hbase:meta 2023-07-16 21:15:40,384 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(421): Removing 'Group_testCreateAndAssign' descriptor. 2023-07-16 21:15:40,386 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=20, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-16 21:15:40,386 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(411): Removing 'Group_testCreateAndAssign' from region states. 2023-07-16 21:15:40,386 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542140386"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:40,389 INFO [PEWorker-2] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-16 21:15:40,389 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e39fdb9ab9ff035bebdf3f7d19e4fb20, NAME => 'Group_testCreateAndAssign,,1689542139294.e39fdb9ab9ff035bebdf3f7d19e4fb20.', STARTKEY => '', ENDKEY => ''}] 2023-07-16 21:15:40,389 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(415): Marking 'Group_testCreateAndAssign' as deleted. 2023-07-16 21:15:40,389 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689542140389"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:40,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=20 2023-07-16 21:15:40,391 INFO [PEWorker-2] hbase.MetaTableAccessor(1658): Deleted table Group_testCreateAndAssign state from META 2023-07-16 21:15:40,394 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(130): Finished pid=20, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-16 21:15:40,396 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=20, state=SUCCESS; DeleteTableProcedure table=Group_testCreateAndAssign in 123 msec 2023-07-16 21:15:40,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=20 2023-07-16 21:15:40,593 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:Group_testCreateAndAssign, procId: 20 completed 2023-07-16 21:15:40,598 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:40,598 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:40,599 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:40,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:40,599 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:40,601 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:40,601 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:40,602 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:40,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:40,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:40,609 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:40,613 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:15:40,614 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:40,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:40,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:40,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:40,620 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:40,624 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:40,624 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:40,627 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:40,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:40,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.CallRunner(144): callId: 163 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:39404 deadline: 1689543340627, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:40,628 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:40,630 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:40,631 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:40,631 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:40,631 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:40,632 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:40,633 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:40,662 INFO [Listener at localhost/43605] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testDefaultNamespaceCreateAndAssign Thread=497 (was 478) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:35261 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741840_1016, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-9 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-8 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:302) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:366) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_628029656_17 at /127.0.0.1:42706 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741840_1016] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741840_1016, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_628029656_17 at /127.0.0.1:53516 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741840_1016] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_628029656_17 at /127.0.0.1:40318 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741840_1016] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-7 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-6 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: AsyncFSWAL-0-hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb-prefix:jenkins-hbase4.apache.org,41213,1689542138767 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-7526253_17 at /127.0.0.1:40332 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741840_1016, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=749 (was 719) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=312 (was 339), ProcessCount=172 (was 172), AvailableMemoryMB=2996 (was 3017) 2023-07-16 21:15:40,682 INFO [Listener at localhost/43605] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testCreateMultiRegion Thread=497, OpenFileDescriptor=749, MaxFileDescriptor=60000, SystemLoadAverage=312, ProcessCount=172, AvailableMemoryMB=2995 2023-07-16 21:15:40,683 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(132): testCreateMultiRegion 2023-07-16 21:15:40,691 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:40,692 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:40,694 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:40,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:40,695 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:40,696 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:40,696 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:40,697 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:40,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:40,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:40,704 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:40,739 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:15:40,741 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:40,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:40,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:40,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:40,750 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:40,755 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:40,755 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:40,759 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:40,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:40,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.CallRunner(144): callId: 191 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:39404 deadline: 1689543340759, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:40,760 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:40,761 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:40,762 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:40,763 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:40,763 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:40,765 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:40,765 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:40,769 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-16 21:15:40,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=21, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=Group_testCreateMultiRegion 2023-07-16 21:15:40,773 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=21, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=Group_testCreateMultiRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-07-16 21:15:40,773 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "Group_testCreateMultiRegion" procId is: 21 2023-07-16 21:15:40,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=21 2023-07-16 21:15:40,777 DEBUG [PEWorker-4] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:40,777 DEBUG [PEWorker-4] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:40,777 DEBUG [PEWorker-4] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:40,784 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=21, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=Group_testCreateMultiRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-16 21:15:40,796 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/3a8b97f1d6bb14f6e3b05cb9b89abdf4 2023-07-16 21:15:40,796 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c7916c21d8a3ef54f6d9fe15de8c3521 2023-07-16 21:15:40,796 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/bc805df06d6faa31e4b1fe04f9136e24 2023-07-16 21:15:40,797 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/3a8b97f1d6bb14f6e3b05cb9b89abdf4 empty. 2023-07-16 21:15:40,797 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e020fe0e78fed6125375355fc8442e1e 2023-07-16 21:15:40,798 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/bc805df06d6faa31e4b1fe04f9136e24 empty. 2023-07-16 21:15:40,798 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/3a8b97f1d6bb14f6e3b05cb9b89abdf4 2023-07-16 21:15:40,798 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/376580fd1a9ca2e27b13627706cff755 2023-07-16 21:15:40,798 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c240c5e2b0a66ca9d8b1dd1f3d696829 2023-07-16 21:15:40,798 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/a0a662f8b472889e1ea9acbed9048152 2023-07-16 21:15:40,798 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e442c5f5ae5f06c0a3bdc2679964b13a 2023-07-16 21:15:40,799 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/cfc44517919312ea044ca4a33874e9d9 2023-07-16 21:15:40,799 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c7916c21d8a3ef54f6d9fe15de8c3521 empty. 2023-07-16 21:15:40,799 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/376580fd1a9ca2e27b13627706cff755 empty. 2023-07-16 21:15:40,800 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/bc805df06d6faa31e4b1fe04f9136e24 2023-07-16 21:15:40,800 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/2dfe6a59994601d58418deb273b19667 2023-07-16 21:15:40,801 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/376580fd1a9ca2e27b13627706cff755 2023-07-16 21:15:40,803 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e020fe0e78fed6125375355fc8442e1e empty. 2023-07-16 21:15:40,803 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/2dfe6a59994601d58418deb273b19667 empty. 2023-07-16 21:15:40,803 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/cfc44517919312ea044ca4a33874e9d9 empty. 2023-07-16 21:15:40,804 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e442c5f5ae5f06c0a3bdc2679964b13a empty. 2023-07-16 21:15:40,804 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e020fe0e78fed6125375355fc8442e1e 2023-07-16 21:15:40,804 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c240c5e2b0a66ca9d8b1dd1f3d696829 empty. 2023-07-16 21:15:40,804 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c7916c21d8a3ef54f6d9fe15de8c3521 2023-07-16 21:15:40,804 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/a0a662f8b472889e1ea9acbed9048152 empty. 2023-07-16 21:15:40,804 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/2dfe6a59994601d58418deb273b19667 2023-07-16 21:15:40,804 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e442c5f5ae5f06c0a3bdc2679964b13a 2023-07-16 21:15:40,805 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c240c5e2b0a66ca9d8b1dd1f3d696829 2023-07-16 21:15:40,804 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/cfc44517919312ea044ca4a33874e9d9 2023-07-16 21:15:40,805 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/a0a662f8b472889e1ea9acbed9048152 2023-07-16 21:15:40,805 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived Group_testCreateMultiRegion regions 2023-07-16 21:15:40,843 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/.tabledesc/.tableinfo.0000000001 2023-07-16 21:15:40,847 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3a8b97f1d6bb14f6e3b05cb9b89abdf4, NAME => 'Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4.', STARTKEY => '', ENDKEY => '\x00\x02\x04\x06\x08'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:40,847 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(7675): creating {ENCODED => c7916c21d8a3ef54f6d9fe15de8c3521, NAME => 'Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521.', STARTKEY => '\x00\x02\x04\x06\x08', ENDKEY => '\x00"$&('}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:40,852 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(7675): creating {ENCODED => bc805df06d6faa31e4b1fe04f9136e24, NAME => 'Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24.', STARTKEY => '\x00"$&(', ENDKEY => '\x00BDFH'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:40,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=21 2023-07-16 21:15:40,909 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:40,912 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1604): Closing bc805df06d6faa31e4b1fe04f9136e24, disabling compactions & flushes 2023-07-16 21:15:40,912 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24. 2023-07-16 21:15:40,913 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24. 2023-07-16 21:15:40,913 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24. after waiting 0 ms 2023-07-16 21:15:40,913 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24. 2023-07-16 21:15:40,913 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24. 2023-07-16 21:15:40,913 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1558): Region close journal for bc805df06d6faa31e4b1fe04f9136e24: 2023-07-16 21:15:40,914 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(7675): creating {ENCODED => e020fe0e78fed6125375355fc8442e1e, NAME => 'Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e.', STARTKEY => '\x00BDFH', ENDKEY => '\x00bdfh'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:40,921 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:40,921 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1604): Closing 3a8b97f1d6bb14f6e3b05cb9b89abdf4, disabling compactions & flushes 2023-07-16 21:15:40,921 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4. 2023-07-16 21:15:40,921 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4. 2023-07-16 21:15:40,921 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4. after waiting 0 ms 2023-07-16 21:15:40,921 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4. 2023-07-16 21:15:40,921 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4. 2023-07-16 21:15:40,921 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1558): Region close journal for 3a8b97f1d6bb14f6e3b05cb9b89abdf4: 2023-07-16 21:15:40,922 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => 376580fd1a9ca2e27b13627706cff755, NAME => 'Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755.', STARTKEY => '\x00bdfh', ENDKEY => '\x00\x82\x84\x86\x88'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:40,923 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:40,924 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1604): Closing c7916c21d8a3ef54f6d9fe15de8c3521, disabling compactions & flushes 2023-07-16 21:15:40,924 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521. 2023-07-16 21:15:40,924 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521. 2023-07-16 21:15:40,924 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521. after waiting 0 ms 2023-07-16 21:15:40,924 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521. 2023-07-16 21:15:40,924 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521. 2023-07-16 21:15:40,924 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1558): Region close journal for c7916c21d8a3ef54f6d9fe15de8c3521: 2023-07-16 21:15:40,925 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(7675): creating {ENCODED => c240c5e2b0a66ca9d8b1dd1f3d696829, NAME => 'Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829.', STARTKEY => '\x00\x82\x84\x86\x88', ENDKEY => '\x00\xA2\xA4\xA6\xA8'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:40,951 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:40,955 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1604): Closing e020fe0e78fed6125375355fc8442e1e, disabling compactions & flushes 2023-07-16 21:15:40,956 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e. 2023-07-16 21:15:40,956 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e. 2023-07-16 21:15:40,956 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e. after waiting 0 ms 2023-07-16 21:15:40,956 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e. 2023-07-16 21:15:40,956 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e. 2023-07-16 21:15:40,957 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1558): Region close journal for e020fe0e78fed6125375355fc8442e1e: 2023-07-16 21:15:40,957 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(7675): creating {ENCODED => e442c5f5ae5f06c0a3bdc2679964b13a, NAME => 'Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a.', STARTKEY => '\x00\xA2\xA4\xA6\xA8', ENDKEY => '\x00\xC2\xC4\xC6\xC8'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:40,970 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:40,971 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1604): Closing 376580fd1a9ca2e27b13627706cff755, disabling compactions & flushes 2023-07-16 21:15:40,971 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755. 2023-07-16 21:15:40,971 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755. 2023-07-16 21:15:40,971 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755. after waiting 0 ms 2023-07-16 21:15:40,971 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755. 2023-07-16 21:15:40,971 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:40,971 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755. 2023-07-16 21:15:40,972 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1604): Closing c240c5e2b0a66ca9d8b1dd1f3d696829, disabling compactions & flushes 2023-07-16 21:15:40,972 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1558): Region close journal for 376580fd1a9ca2e27b13627706cff755: 2023-07-16 21:15:40,972 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829. 2023-07-16 21:15:40,972 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829. 2023-07-16 21:15:40,972 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => a0a662f8b472889e1ea9acbed9048152, NAME => 'Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152.', STARTKEY => '\x00\xC2\xC4\xC6\xC8', ENDKEY => '\x00\xE2\xE4\xE6\xE8'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:40,972 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829. after waiting 0 ms 2023-07-16 21:15:40,973 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829. 2023-07-16 21:15:40,973 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829. 2023-07-16 21:15:40,973 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1558): Region close journal for c240c5e2b0a66ca9d8b1dd1f3d696829: 2023-07-16 21:15:40,973 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(7675): creating {ENCODED => cfc44517919312ea044ca4a33874e9d9, NAME => 'Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9.', STARTKEY => '\x00\xE2\xE4\xE6\xE8', ENDKEY => '\x01\x03\x05\x07\x09'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:40,994 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:40,995 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1604): Closing e442c5f5ae5f06c0a3bdc2679964b13a, disabling compactions & flushes 2023-07-16 21:15:40,995 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a. 2023-07-16 21:15:40,996 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a. 2023-07-16 21:15:40,996 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a. after waiting 0 ms 2023-07-16 21:15:40,996 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a. 2023-07-16 21:15:40,996 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a. 2023-07-16 21:15:40,996 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1558): Region close journal for e442c5f5ae5f06c0a3bdc2679964b13a: 2023-07-16 21:15:40,996 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(7675): creating {ENCODED => 2dfe6a59994601d58418deb273b19667, NAME => 'Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667.', STARTKEY => '\x01\x03\x05\x07\x09', ENDKEY => ''}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:41,023 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:41,024 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:41,026 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1604): Closing cfc44517919312ea044ca4a33874e9d9, disabling compactions & flushes 2023-07-16 21:15:41,026 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1604): Closing a0a662f8b472889e1ea9acbed9048152, disabling compactions & flushes 2023-07-16 21:15:41,026 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9. 2023-07-16 21:15:41,026 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152. 2023-07-16 21:15:41,026 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9. 2023-07-16 21:15:41,026 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152. 2023-07-16 21:15:41,026 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9. after waiting 0 ms 2023-07-16 21:15:41,026 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152. after waiting 0 ms 2023-07-16 21:15:41,026 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9. 2023-07-16 21:15:41,026 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152. 2023-07-16 21:15:41,026 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9. 2023-07-16 21:15:41,026 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152. 2023-07-16 21:15:41,026 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1558): Region close journal for cfc44517919312ea044ca4a33874e9d9: 2023-07-16 21:15:41,026 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1558): Region close journal for a0a662f8b472889e1ea9acbed9048152: 2023-07-16 21:15:41,029 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:41,029 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1604): Closing 2dfe6a59994601d58418deb273b19667, disabling compactions & flushes 2023-07-16 21:15:41,029 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667. 2023-07-16 21:15:41,029 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667. 2023-07-16 21:15:41,029 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667. after waiting 0 ms 2023-07-16 21:15:41,029 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667. 2023-07-16 21:15:41,029 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667. 2023-07-16 21:15:41,029 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1558): Region close journal for 2dfe6a59994601d58418deb273b19667: 2023-07-16 21:15:41,032 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=21, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=Group_testCreateMultiRegion execute state=CREATE_TABLE_ADD_TO_META 2023-07-16 21:15:41,034 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\"$\u0026(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141033"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542141033"}]},"ts":"1689542141033"} 2023-07-16 21:15:41,034 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689542141033"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542141033"}]},"ts":"1689542141033"} 2023-07-16 21:15:41,034 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\x02\\x04\\x06\\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141033"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542141033"}]},"ts":"1689542141033"} 2023-07-16 21:15:41,034 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141033"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542141033"}]},"ts":"1689542141033"} 2023-07-16 21:15:41,034 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141033"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542141033"}]},"ts":"1689542141033"} 2023-07-16 21:15:41,034 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\x82\\x84\\x86\\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141033"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542141033"}]},"ts":"1689542141033"} 2023-07-16 21:15:41,034 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\xA2\\xA4\\xA6\\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141033"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542141033"}]},"ts":"1689542141033"} 2023-07-16 21:15:41,034 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\xE2\\xE4\\xE6\\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141033"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542141033"}]},"ts":"1689542141033"} 2023-07-16 21:15:41,035 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\xC2\\xC4\\xC6\\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141033"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542141033"}]},"ts":"1689542141033"} 2023-07-16 21:15:41,035 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x01\\x03\\x05\\x07\\x09,1689542140769.2dfe6a59994601d58418deb273b19667.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689542141033"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542141033"}]},"ts":"1689542141033"} 2023-07-16 21:15:41,039 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 10 regions to meta. 2023-07-16 21:15:41,040 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=21, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=Group_testCreateMultiRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-16 21:15:41,041 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateMultiRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542141040"}]},"ts":"1689542141040"} 2023-07-16 21:15:41,042 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateMultiRegion, state=ENABLING in hbase:meta 2023-07-16 21:15:41,046 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-16 21:15:41,047 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-16 21:15:41,047 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-16 21:15:41,047 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-16 21:15:41,047 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-16 21:15:41,047 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-16 21:15:41,048 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=3a8b97f1d6bb14f6e3b05cb9b89abdf4, ASSIGN}, {pid=23, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=c7916c21d8a3ef54f6d9fe15de8c3521, ASSIGN}, {pid=24, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=bc805df06d6faa31e4b1fe04f9136e24, ASSIGN}, {pid=25, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=e020fe0e78fed6125375355fc8442e1e, ASSIGN}, {pid=26, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=376580fd1a9ca2e27b13627706cff755, ASSIGN}, {pid=27, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=c240c5e2b0a66ca9d8b1dd1f3d696829, ASSIGN}, {pid=28, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=e442c5f5ae5f06c0a3bdc2679964b13a, ASSIGN}, {pid=29, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=a0a662f8b472889e1ea9acbed9048152, ASSIGN}, {pid=30, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=cfc44517919312ea044ca4a33874e9d9, ASSIGN}, {pid=31, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=2dfe6a59994601d58418deb273b19667, ASSIGN}] 2023-07-16 21:15:41,051 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=3a8b97f1d6bb14f6e3b05cb9b89abdf4, ASSIGN 2023-07-16 21:15:41,051 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=c7916c21d8a3ef54f6d9fe15de8c3521, ASSIGN 2023-07-16 21:15:41,052 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=bc805df06d6faa31e4b1fe04f9136e24, ASSIGN 2023-07-16 21:15:41,052 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=25, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=e020fe0e78fed6125375355fc8442e1e, ASSIGN 2023-07-16 21:15:41,054 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=c7916c21d8a3ef54f6d9fe15de8c3521, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41213,1689542138767; forceNewPlan=false, retain=false 2023-07-16 21:15:41,054 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=25, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=e020fe0e78fed6125375355fc8442e1e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45229,1689542134233; forceNewPlan=false, retain=false 2023-07-16 21:15:41,054 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=2dfe6a59994601d58418deb273b19667, ASSIGN 2023-07-16 21:15:41,054 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=bc805df06d6faa31e4b1fe04f9136e24, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33511,1689542134438; forceNewPlan=false, retain=false 2023-07-16 21:15:41,055 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=3a8b97f1d6bb14f6e3b05cb9b89abdf4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36993,1689542134044; forceNewPlan=false, retain=false 2023-07-16 21:15:41,056 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=cfc44517919312ea044ca4a33874e9d9, ASSIGN 2023-07-16 21:15:41,056 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=a0a662f8b472889e1ea9acbed9048152, ASSIGN 2023-07-16 21:15:41,056 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=2dfe6a59994601d58418deb273b19667, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36993,1689542134044; forceNewPlan=false, retain=false 2023-07-16 21:15:41,056 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=e442c5f5ae5f06c0a3bdc2679964b13a, ASSIGN 2023-07-16 21:15:41,057 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=27, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=c240c5e2b0a66ca9d8b1dd1f3d696829, ASSIGN 2023-07-16 21:15:41,058 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=cfc44517919312ea044ca4a33874e9d9, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45229,1689542134233; forceNewPlan=false, retain=false 2023-07-16 21:15:41,059 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=e442c5f5ae5f06c0a3bdc2679964b13a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41213,1689542138767; forceNewPlan=false, retain=false 2023-07-16 21:15:41,059 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=27, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=c240c5e2b0a66ca9d8b1dd1f3d696829, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36993,1689542134044; forceNewPlan=false, retain=false 2023-07-16 21:15:41,059 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=376580fd1a9ca2e27b13627706cff755, ASSIGN 2023-07-16 21:15:41,058 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=a0a662f8b472889e1ea9acbed9048152, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33511,1689542134438; forceNewPlan=false, retain=false 2023-07-16 21:15:41,060 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=376580fd1a9ca2e27b13627706cff755, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33511,1689542134438; forceNewPlan=false, retain=false 2023-07-16 21:15:41,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=21 2023-07-16 21:15:41,205 INFO [jenkins-hbase4:41499] balancer.BaseLoadBalancer(1545): Reassigned 10 regions. 10 retained the pre-restart assignment. 2023-07-16 21:15:41,210 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=c7916c21d8a3ef54f6d9fe15de8c3521, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:41,210 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=e442c5f5ae5f06c0a3bdc2679964b13a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:41,210 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=25 updating hbase:meta row=e020fe0e78fed6125375355fc8442e1e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:41,210 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=2dfe6a59994601d58418deb273b19667, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:41,210 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\x02\\x04\\x06\\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141210"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141210"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141210"}]},"ts":"1689542141210"} 2023-07-16 21:15:41,210 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141210"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141210"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141210"}]},"ts":"1689542141210"} 2023-07-16 21:15:41,210 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=cfc44517919312ea044ca4a33874e9d9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:41,210 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x01\\x03\\x05\\x07\\x09,1689542140769.2dfe6a59994601d58418deb273b19667.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689542141210"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141210"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141210"}]},"ts":"1689542141210"} 2023-07-16 21:15:41,210 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\xA2\\xA4\\xA6\\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141210"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141210"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141210"}]},"ts":"1689542141210"} 2023-07-16 21:15:41,210 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\xE2\\xE4\\xE6\\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141210"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141210"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141210"}]},"ts":"1689542141210"} 2023-07-16 21:15:41,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=25, state=RUNNABLE; OpenRegionProcedure e020fe0e78fed6125375355fc8442e1e, server=jenkins-hbase4.apache.org,45229,1689542134233}] 2023-07-16 21:15:41,214 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=23, state=RUNNABLE; OpenRegionProcedure c7916c21d8a3ef54f6d9fe15de8c3521, server=jenkins-hbase4.apache.org,41213,1689542138767}] 2023-07-16 21:15:41,214 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure 2dfe6a59994601d58418deb273b19667, server=jenkins-hbase4.apache.org,36993,1689542134044}] 2023-07-16 21:15:41,215 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=35, ppid=28, state=RUNNABLE; OpenRegionProcedure e442c5f5ae5f06c0a3bdc2679964b13a, server=jenkins-hbase4.apache.org,41213,1689542138767}] 2023-07-16 21:15:41,217 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=27 updating hbase:meta row=c240c5e2b0a66ca9d8b1dd1f3d696829, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:41,217 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\x82\\x84\\x86\\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141217"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141217"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141217"}]},"ts":"1689542141217"} 2023-07-16 21:15:41,217 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=30, state=RUNNABLE; OpenRegionProcedure cfc44517919312ea044ca4a33874e9d9, server=jenkins-hbase4.apache.org,45229,1689542134233}] 2023-07-16 21:15:41,220 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=3a8b97f1d6bb14f6e3b05cb9b89abdf4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:41,220 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689542141220"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141220"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141220"}]},"ts":"1689542141220"} 2023-07-16 21:15:41,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=27, state=RUNNABLE; OpenRegionProcedure c240c5e2b0a66ca9d8b1dd1f3d696829, server=jenkins-hbase4.apache.org,36993,1689542134044}] 2023-07-16 21:15:41,221 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=a0a662f8b472889e1ea9acbed9048152, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:41,221 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\xC2\\xC4\\xC6\\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141221"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141221"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141221"}]},"ts":"1689542141221"} 2023-07-16 21:15:41,222 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=376580fd1a9ca2e27b13627706cff755, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:41,222 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=22, state=RUNNABLE; OpenRegionProcedure 3a8b97f1d6bb14f6e3b05cb9b89abdf4, server=jenkins-hbase4.apache.org,36993,1689542134044}] 2023-07-16 21:15:41,222 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141222"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141222"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141222"}]},"ts":"1689542141222"} 2023-07-16 21:15:41,223 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=bc805df06d6faa31e4b1fe04f9136e24, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:41,223 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\"$\u0026(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141223"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141223"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141223"}]},"ts":"1689542141223"} 2023-07-16 21:15:41,224 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=29, state=RUNNABLE; OpenRegionProcedure a0a662f8b472889e1ea9acbed9048152, server=jenkins-hbase4.apache.org,33511,1689542134438}] 2023-07-16 21:15:41,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=26, state=RUNNABLE; OpenRegionProcedure 376580fd1a9ca2e27b13627706cff755, server=jenkins-hbase4.apache.org,33511,1689542134438}] 2023-07-16 21:15:41,226 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=24, state=RUNNABLE; OpenRegionProcedure bc805df06d6faa31e4b1fe04f9136e24, server=jenkins-hbase4.apache.org,33511,1689542134438}] 2023-07-16 21:15:41,366 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:41,366 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-16 21:15:41,370 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53434, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-16 21:15:41,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e. 2023-07-16 21:15:41,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e020fe0e78fed6125375355fc8442e1e, NAME => 'Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e.', STARTKEY => '\x00BDFH', ENDKEY => '\x00bdfh'} 2023-07-16 21:15:41,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion e020fe0e78fed6125375355fc8442e1e 2023-07-16 21:15:41,377 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:41,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:41,378 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-16 21:15:41,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e020fe0e78fed6125375355fc8442e1e 2023-07-16 21:15:41,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e020fe0e78fed6125375355fc8442e1e 2023-07-16 21:15:41,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=21 2023-07-16 21:15:41,419 INFO [StoreOpener-e020fe0e78fed6125375355fc8442e1e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region e020fe0e78fed6125375355fc8442e1e 2023-07-16 21:15:41,430 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39144, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-16 21:15:41,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4. 2023-07-16 21:15:41,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3a8b97f1d6bb14f6e3b05cb9b89abdf4, NAME => 'Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4.', STARTKEY => '', ENDKEY => '\x00\x02\x04\x06\x08'} 2023-07-16 21:15:41,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion 3a8b97f1d6bb14f6e3b05cb9b89abdf4 2023-07-16 21:15:41,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:41,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3a8b97f1d6bb14f6e3b05cb9b89abdf4 2023-07-16 21:15:41,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3a8b97f1d6bb14f6e3b05cb9b89abdf4 2023-07-16 21:15:41,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521. 2023-07-16 21:15:41,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c7916c21d8a3ef54f6d9fe15de8c3521, NAME => 'Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521.', STARTKEY => '\x00\x02\x04\x06\x08', ENDKEY => '\x00"$&('} 2023-07-16 21:15:41,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion c7916c21d8a3ef54f6d9fe15de8c3521 2023-07-16 21:15:41,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:41,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c7916c21d8a3ef54f6d9fe15de8c3521 2023-07-16 21:15:41,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c7916c21d8a3ef54f6d9fe15de8c3521 2023-07-16 21:15:41,437 DEBUG [StoreOpener-e020fe0e78fed6125375355fc8442e1e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/e020fe0e78fed6125375355fc8442e1e/f 2023-07-16 21:15:41,438 DEBUG [StoreOpener-e020fe0e78fed6125375355fc8442e1e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/e020fe0e78fed6125375355fc8442e1e/f 2023-07-16 21:15:41,438 INFO [StoreOpener-e020fe0e78fed6125375355fc8442e1e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e020fe0e78fed6125375355fc8442e1e columnFamilyName f 2023-07-16 21:15:41,439 INFO [StoreOpener-e020fe0e78fed6125375355fc8442e1e-1] regionserver.HStore(310): Store=e020fe0e78fed6125375355fc8442e1e/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:41,450 INFO [StoreOpener-3a8b97f1d6bb14f6e3b05cb9b89abdf4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region 3a8b97f1d6bb14f6e3b05cb9b89abdf4 2023-07-16 21:15:41,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24. 2023-07-16 21:15:41,459 INFO [StoreOpener-c7916c21d8a3ef54f6d9fe15de8c3521-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region c7916c21d8a3ef54f6d9fe15de8c3521 2023-07-16 21:15:41,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bc805df06d6faa31e4b1fe04f9136e24, NAME => 'Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24.', STARTKEY => '\x00"$&(', ENDKEY => '\x00BDFH'} 2023-07-16 21:15:41,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion bc805df06d6faa31e4b1fe04f9136e24 2023-07-16 21:15:41,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:41,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/e020fe0e78fed6125375355fc8442e1e 2023-07-16 21:15:41,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bc805df06d6faa31e4b1fe04f9136e24 2023-07-16 21:15:41,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bc805df06d6faa31e4b1fe04f9136e24 2023-07-16 21:15:41,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/e020fe0e78fed6125375355fc8442e1e 2023-07-16 21:15:41,462 DEBUG [StoreOpener-3a8b97f1d6bb14f6e3b05cb9b89abdf4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/3a8b97f1d6bb14f6e3b05cb9b89abdf4/f 2023-07-16 21:15:41,462 DEBUG [StoreOpener-3a8b97f1d6bb14f6e3b05cb9b89abdf4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/3a8b97f1d6bb14f6e3b05cb9b89abdf4/f 2023-07-16 21:15:41,463 INFO [StoreOpener-3a8b97f1d6bb14f6e3b05cb9b89abdf4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3a8b97f1d6bb14f6e3b05cb9b89abdf4 columnFamilyName f 2023-07-16 21:15:41,464 INFO [StoreOpener-3a8b97f1d6bb14f6e3b05cb9b89abdf4-1] regionserver.HStore(310): Store=3a8b97f1d6bb14f6e3b05cb9b89abdf4/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:41,464 DEBUG [StoreOpener-c7916c21d8a3ef54f6d9fe15de8c3521-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/c7916c21d8a3ef54f6d9fe15de8c3521/f 2023-07-16 21:15:41,474 DEBUG [StoreOpener-c7916c21d8a3ef54f6d9fe15de8c3521-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/c7916c21d8a3ef54f6d9fe15de8c3521/f 2023-07-16 21:15:41,474 INFO [StoreOpener-bc805df06d6faa31e4b1fe04f9136e24-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region bc805df06d6faa31e4b1fe04f9136e24 2023-07-16 21:15:41,475 INFO [StoreOpener-c7916c21d8a3ef54f6d9fe15de8c3521-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c7916c21d8a3ef54f6d9fe15de8c3521 columnFamilyName f 2023-07-16 21:15:41,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/3a8b97f1d6bb14f6e3b05cb9b89abdf4 2023-07-16 21:15:41,476 INFO [StoreOpener-c7916c21d8a3ef54f6d9fe15de8c3521-1] regionserver.HStore(310): Store=c7916c21d8a3ef54f6d9fe15de8c3521/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:41,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/3a8b97f1d6bb14f6e3b05cb9b89abdf4 2023-07-16 21:15:41,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/c7916c21d8a3ef54f6d9fe15de8c3521 2023-07-16 21:15:41,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e020fe0e78fed6125375355fc8442e1e 2023-07-16 21:15:41,479 DEBUG [StoreOpener-bc805df06d6faa31e4b1fe04f9136e24-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/bc805df06d6faa31e4b1fe04f9136e24/f 2023-07-16 21:15:41,479 DEBUG [StoreOpener-bc805df06d6faa31e4b1fe04f9136e24-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/bc805df06d6faa31e4b1fe04f9136e24/f 2023-07-16 21:15:41,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/c7916c21d8a3ef54f6d9fe15de8c3521 2023-07-16 21:15:41,481 INFO [StoreOpener-bc805df06d6faa31e4b1fe04f9136e24-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bc805df06d6faa31e4b1fe04f9136e24 columnFamilyName f 2023-07-16 21:15:41,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3a8b97f1d6bb14f6e3b05cb9b89abdf4 2023-07-16 21:15:41,483 INFO [StoreOpener-bc805df06d6faa31e4b1fe04f9136e24-1] regionserver.HStore(310): Store=bc805df06d6faa31e4b1fe04f9136e24/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:41,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/e020fe0e78fed6125375355fc8442e1e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:41,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e020fe0e78fed6125375355fc8442e1e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11261051520, jitterRate=0.048767149448394775}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:41,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e020fe0e78fed6125375355fc8442e1e: 2023-07-16 21:15:41,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/bc805df06d6faa31e4b1fe04f9136e24 2023-07-16 21:15:41,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e., pid=32, masterSystemTime=1689542141365 2023-07-16 21:15:41,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/bc805df06d6faa31e4b1fe04f9136e24 2023-07-16 21:15:41,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/3a8b97f1d6bb14f6e3b05cb9b89abdf4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:41,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e. 2023-07-16 21:15:41,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e. 2023-07-16 21:15:41,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9. 2023-07-16 21:15:41,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cfc44517919312ea044ca4a33874e9d9, NAME => 'Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9.', STARTKEY => '\x00\xE2\xE4\xE6\xE8', ENDKEY => '\x01\x03\x05\x07\x09'} 2023-07-16 21:15:41,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion cfc44517919312ea044ca4a33874e9d9 2023-07-16 21:15:41,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:41,489 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=25 updating hbase:meta row=e020fe0e78fed6125375355fc8442e1e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:41,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for cfc44517919312ea044ca4a33874e9d9 2023-07-16 21:15:41,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for cfc44517919312ea044ca4a33874e9d9 2023-07-16 21:15:41,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3a8b97f1d6bb14f6e3b05cb9b89abdf4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=9719603040, jitterRate=-0.09479142725467682}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:41,489 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141488"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542141488"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542141488"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542141488"}]},"ts":"1689542141488"} 2023-07-16 21:15:41,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3a8b97f1d6bb14f6e3b05cb9b89abdf4: 2023-07-16 21:15:41,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c7916c21d8a3ef54f6d9fe15de8c3521 2023-07-16 21:15:41,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4., pid=38, masterSystemTime=1689542141368 2023-07-16 21:15:41,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4. 2023-07-16 21:15:41,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4. 2023-07-16 21:15:41,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667. 2023-07-16 21:15:41,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2dfe6a59994601d58418deb273b19667, NAME => 'Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667.', STARTKEY => '\x01\x03\x05\x07\x09', ENDKEY => ''} 2023-07-16 21:15:41,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion 2dfe6a59994601d58418deb273b19667 2023-07-16 21:15:41,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:41,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2dfe6a59994601d58418deb273b19667 2023-07-16 21:15:41,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2dfe6a59994601d58418deb273b19667 2023-07-16 21:15:41,497 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=3a8b97f1d6bb14f6e3b05cb9b89abdf4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:41,497 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689542141497"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542141497"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542141497"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542141497"}]},"ts":"1689542141497"} 2023-07-16 21:15:41,498 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=25 2023-07-16 21:15:41,498 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=25, state=SUCCESS; OpenRegionProcedure e020fe0e78fed6125375355fc8442e1e, server=jenkins-hbase4.apache.org,45229,1689542134233 in 280 msec 2023-07-16 21:15:41,500 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=21, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=e020fe0e78fed6125375355fc8442e1e, ASSIGN in 450 msec 2023-07-16 21:15:41,502 INFO [StoreOpener-cfc44517919312ea044ca4a33874e9d9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region cfc44517919312ea044ca4a33874e9d9 2023-07-16 21:15:41,502 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=22 2023-07-16 21:15:41,503 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=22, state=SUCCESS; OpenRegionProcedure 3a8b97f1d6bb14f6e3b05cb9b89abdf4, server=jenkins-hbase4.apache.org,36993,1689542134044 in 277 msec 2023-07-16 21:15:41,505 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=3a8b97f1d6bb14f6e3b05cb9b89abdf4, ASSIGN in 456 msec 2023-07-16 21:15:41,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bc805df06d6faa31e4b1fe04f9136e24 2023-07-16 21:15:41,513 INFO [StoreOpener-2dfe6a59994601d58418deb273b19667-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region 2dfe6a59994601d58418deb273b19667 2023-07-16 21:15:41,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/c7916c21d8a3ef54f6d9fe15de8c3521/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:41,517 DEBUG [StoreOpener-cfc44517919312ea044ca4a33874e9d9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/cfc44517919312ea044ca4a33874e9d9/f 2023-07-16 21:15:41,517 DEBUG [StoreOpener-cfc44517919312ea044ca4a33874e9d9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/cfc44517919312ea044ca4a33874e9d9/f 2023-07-16 21:15:41,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c7916c21d8a3ef54f6d9fe15de8c3521; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11505772960, jitterRate=0.07155860960483551}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:41,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c7916c21d8a3ef54f6d9fe15de8c3521: 2023-07-16 21:15:41,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/bc805df06d6faa31e4b1fe04f9136e24/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:41,518 INFO [StoreOpener-cfc44517919312ea044ca4a33874e9d9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cfc44517919312ea044ca4a33874e9d9 columnFamilyName f 2023-07-16 21:15:41,518 DEBUG [StoreOpener-2dfe6a59994601d58418deb273b19667-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/2dfe6a59994601d58418deb273b19667/f 2023-07-16 21:15:41,518 DEBUG [StoreOpener-2dfe6a59994601d58418deb273b19667-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/2dfe6a59994601d58418deb273b19667/f 2023-07-16 21:15:41,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bc805df06d6faa31e4b1fe04f9136e24; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10613520800, jitterRate=-0.011538848280906677}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:41,520 INFO [StoreOpener-cfc44517919312ea044ca4a33874e9d9-1] regionserver.HStore(310): Store=cfc44517919312ea044ca4a33874e9d9/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:41,520 INFO [StoreOpener-2dfe6a59994601d58418deb273b19667-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2dfe6a59994601d58418deb273b19667 columnFamilyName f 2023-07-16 21:15:41,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521., pid=33, masterSystemTime=1689542141366 2023-07-16 21:15:41,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bc805df06d6faa31e4b1fe04f9136e24: 2023-07-16 21:15:41,523 INFO [StoreOpener-2dfe6a59994601d58418deb273b19667-1] regionserver.HStore(310): Store=2dfe6a59994601d58418deb273b19667/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:41,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/cfc44517919312ea044ca4a33874e9d9 2023-07-16 21:15:41,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/cfc44517919312ea044ca4a33874e9d9 2023-07-16 21:15:41,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/2dfe6a59994601d58418deb273b19667 2023-07-16 21:15:41,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/2dfe6a59994601d58418deb273b19667 2023-07-16 21:15:41,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24., pid=41, masterSystemTime=1689542141377 2023-07-16 21:15:41,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521. 2023-07-16 21:15:41,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521. 2023-07-16 21:15:41,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a. 2023-07-16 21:15:41,533 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=c7916c21d8a3ef54f6d9fe15de8c3521, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:41,533 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00\\x02\\x04\\x06\\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141532"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542141532"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542141532"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542141532"}]},"ts":"1689542141532"} 2023-07-16 21:15:41,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e442c5f5ae5f06c0a3bdc2679964b13a, NAME => 'Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a.', STARTKEY => '\x00\xA2\xA4\xA6\xA8', ENDKEY => '\x00\xC2\xC4\xC6\xC8'} 2023-07-16 21:15:41,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion e442c5f5ae5f06c0a3bdc2679964b13a 2023-07-16 21:15:41,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:41,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e442c5f5ae5f06c0a3bdc2679964b13a 2023-07-16 21:15:41,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e442c5f5ae5f06c0a3bdc2679964b13a 2023-07-16 21:15:41,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for cfc44517919312ea044ca4a33874e9d9 2023-07-16 21:15:41,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24. 2023-07-16 21:15:41,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24. 2023-07-16 21:15:41,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755. 2023-07-16 21:15:41,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 376580fd1a9ca2e27b13627706cff755, NAME => 'Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755.', STARTKEY => '\x00bdfh', ENDKEY => '\x00\x82\x84\x86\x88'} 2023-07-16 21:15:41,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion 376580fd1a9ca2e27b13627706cff755 2023-07-16 21:15:41,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:41,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 376580fd1a9ca2e27b13627706cff755 2023-07-16 21:15:41,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 376580fd1a9ca2e27b13627706cff755 2023-07-16 21:15:41,539 INFO [StoreOpener-e442c5f5ae5f06c0a3bdc2679964b13a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region e442c5f5ae5f06c0a3bdc2679964b13a 2023-07-16 21:15:41,576 INFO [StoreOpener-376580fd1a9ca2e27b13627706cff755-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region 376580fd1a9ca2e27b13627706cff755 2023-07-16 21:15:41,577 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=bc805df06d6faa31e4b1fe04f9136e24, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:41,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2dfe6a59994601d58418deb273b19667 2023-07-16 21:15:41,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/cfc44517919312ea044ca4a33874e9d9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:41,582 DEBUG [StoreOpener-376580fd1a9ca2e27b13627706cff755-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/376580fd1a9ca2e27b13627706cff755/f 2023-07-16 21:15:41,582 DEBUG [StoreOpener-376580fd1a9ca2e27b13627706cff755-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/376580fd1a9ca2e27b13627706cff755/f 2023-07-16 21:15:41,582 DEBUG [StoreOpener-e442c5f5ae5f06c0a3bdc2679964b13a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/e442c5f5ae5f06c0a3bdc2679964b13a/f 2023-07-16 21:15:41,582 DEBUG [StoreOpener-e442c5f5ae5f06c0a3bdc2679964b13a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/e442c5f5ae5f06c0a3bdc2679964b13a/f 2023-07-16 21:15:41,583 INFO [StoreOpener-376580fd1a9ca2e27b13627706cff755-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 376580fd1a9ca2e27b13627706cff755 columnFamilyName f 2023-07-16 21:15:41,583 INFO [StoreOpener-e442c5f5ae5f06c0a3bdc2679964b13a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e442c5f5ae5f06c0a3bdc2679964b13a columnFamilyName f 2023-07-16 21:15:41,584 INFO [StoreOpener-376580fd1a9ca2e27b13627706cff755-1] regionserver.HStore(310): Store=376580fd1a9ca2e27b13627706cff755/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:41,585 INFO [StoreOpener-e442c5f5ae5f06c0a3bdc2679964b13a-1] regionserver.HStore(310): Store=e442c5f5ae5f06c0a3bdc2679964b13a/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:41,585 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=23 2023-07-16 21:15:41,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=23, state=SUCCESS; OpenRegionProcedure c7916c21d8a3ef54f6d9fe15de8c3521, server=jenkins-hbase4.apache.org,41213,1689542138767 in 326 msec 2023-07-16 21:15:41,587 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00\"$\u0026(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141540"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542141540"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542141540"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542141540"}]},"ts":"1689542141540"} 2023-07-16 21:15:41,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened cfc44517919312ea044ca4a33874e9d9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10199241920, jitterRate=-0.050121575593948364}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:41,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for cfc44517919312ea044ca4a33874e9d9: 2023-07-16 21:15:41,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/e442c5f5ae5f06c0a3bdc2679964b13a 2023-07-16 21:15:41,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/376580fd1a9ca2e27b13627706cff755 2023-07-16 21:15:41,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9., pid=36, masterSystemTime=1689542141365 2023-07-16 21:15:41,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/376580fd1a9ca2e27b13627706cff755 2023-07-16 21:15:41,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/e442c5f5ae5f06c0a3bdc2679964b13a 2023-07-16 21:15:41,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9. 2023-07-16 21:15:41,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9. 2023-07-16 21:15:41,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 376580fd1a9ca2e27b13627706cff755 2023-07-16 21:15:41,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e442c5f5ae5f06c0a3bdc2679964b13a 2023-07-16 21:15:41,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/2dfe6a59994601d58418deb273b19667/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:41,601 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=24 2023-07-16 21:15:41,601 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=24, state=SUCCESS; OpenRegionProcedure bc805df06d6faa31e4b1fe04f9136e24, server=jenkins-hbase4.apache.org,33511,1689542134438 in 365 msec 2023-07-16 21:15:41,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=21, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=c7916c21d8a3ef54f6d9fe15de8c3521, ASSIGN in 539 msec 2023-07-16 21:15:41,601 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=cfc44517919312ea044ca4a33874e9d9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:41,602 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00\\xE2\\xE4\\xE6\\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141601"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542141601"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542141601"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542141601"}]},"ts":"1689542141601"} 2023-07-16 21:15:41,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2dfe6a59994601d58418deb273b19667; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11735138080, jitterRate=0.09291990101337433}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:41,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2dfe6a59994601d58418deb273b19667: 2023-07-16 21:15:41,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667., pid=34, masterSystemTime=1689542141368 2023-07-16 21:15:41,604 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=21, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=bc805df06d6faa31e4b1fe04f9136e24, ASSIGN in 553 msec 2023-07-16 21:15:41,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667. 2023-07-16 21:15:41,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667. 2023-07-16 21:15:41,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829. 2023-07-16 21:15:41,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c240c5e2b0a66ca9d8b1dd1f3d696829, NAME => 'Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829.', STARTKEY => '\x00\x82\x84\x86\x88', ENDKEY => '\x00\xA2\xA4\xA6\xA8'} 2023-07-16 21:15:41,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion c240c5e2b0a66ca9d8b1dd1f3d696829 2023-07-16 21:15:41,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:41,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c240c5e2b0a66ca9d8b1dd1f3d696829 2023-07-16 21:15:41,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c240c5e2b0a66ca9d8b1dd1f3d696829 2023-07-16 21:15:41,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/376580fd1a9ca2e27b13627706cff755/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:41,608 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=2dfe6a59994601d58418deb273b19667, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:41,608 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x01\\x03\\x05\\x07\\x09,1689542140769.2dfe6a59994601d58418deb273b19667.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689542141608"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542141608"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542141608"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542141608"}]},"ts":"1689542141608"} 2023-07-16 21:15:41,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/e442c5f5ae5f06c0a3bdc2679964b13a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:41,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 376580fd1a9ca2e27b13627706cff755; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10914290400, jitterRate=0.016472503542900085}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:41,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 376580fd1a9ca2e27b13627706cff755: 2023-07-16 21:15:41,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e442c5f5ae5f06c0a3bdc2679964b13a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11518266560, jitterRate=0.0727221667766571}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:41,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e442c5f5ae5f06c0a3bdc2679964b13a: 2023-07-16 21:15:41,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755., pid=40, masterSystemTime=1689542141377 2023-07-16 21:15:41,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a., pid=35, masterSystemTime=1689542141366 2023-07-16 21:15:41,613 INFO [StoreOpener-c240c5e2b0a66ca9d8b1dd1f3d696829-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region c240c5e2b0a66ca9d8b1dd1f3d696829 2023-07-16 21:15:41,614 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=30 2023-07-16 21:15:41,614 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=30, state=SUCCESS; OpenRegionProcedure cfc44517919312ea044ca4a33874e9d9, server=jenkins-hbase4.apache.org,45229,1689542134233 in 387 msec 2023-07-16 21:15:41,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755. 2023-07-16 21:15:41,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755. 2023-07-16 21:15:41,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152. 2023-07-16 21:15:41,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a0a662f8b472889e1ea9acbed9048152, NAME => 'Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152.', STARTKEY => '\x00\xC2\xC4\xC6\xC8', ENDKEY => '\x00\xE2\xE4\xE6\xE8'} 2023-07-16 21:15:41,615 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=376580fd1a9ca2e27b13627706cff755, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:41,616 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion a0a662f8b472889e1ea9acbed9048152 2023-07-16 21:15:41,616 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:41,616 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a0a662f8b472889e1ea9acbed9048152 2023-07-16 21:15:41,616 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a0a662f8b472889e1ea9acbed9048152 2023-07-16 21:15:41,616 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a. 2023-07-16 21:15:41,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a. 2023-07-16 21:15:41,617 DEBUG [StoreOpener-c240c5e2b0a66ca9d8b1dd1f3d696829-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/c240c5e2b0a66ca9d8b1dd1f3d696829/f 2023-07-16 21:15:41,618 DEBUG [StoreOpener-c240c5e2b0a66ca9d8b1dd1f3d696829-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/c240c5e2b0a66ca9d8b1dd1f3d696829/f 2023-07-16 21:15:41,618 INFO [StoreOpener-c240c5e2b0a66ca9d8b1dd1f3d696829-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c240c5e2b0a66ca9d8b1dd1f3d696829 columnFamilyName f 2023-07-16 21:15:41,619 INFO [StoreOpener-a0a662f8b472889e1ea9acbed9048152-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region a0a662f8b472889e1ea9acbed9048152 2023-07-16 21:15:41,619 INFO [StoreOpener-c240c5e2b0a66ca9d8b1dd1f3d696829-1] regionserver.HStore(310): Store=c240c5e2b0a66ca9d8b1dd1f3d696829/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:41,620 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141615"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542141615"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542141615"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542141615"}]},"ts":"1689542141615"} 2023-07-16 21:15:41,621 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/c240c5e2b0a66ca9d8b1dd1f3d696829 2023-07-16 21:15:41,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/c240c5e2b0a66ca9d8b1dd1f3d696829 2023-07-16 21:15:41,622 DEBUG [StoreOpener-a0a662f8b472889e1ea9acbed9048152-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/a0a662f8b472889e1ea9acbed9048152/f 2023-07-16 21:15:41,622 DEBUG [StoreOpener-a0a662f8b472889e1ea9acbed9048152-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/a0a662f8b472889e1ea9acbed9048152/f 2023-07-16 21:15:41,623 INFO [StoreOpener-a0a662f8b472889e1ea9acbed9048152-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a0a662f8b472889e1ea9acbed9048152 columnFamilyName f 2023-07-16 21:15:41,626 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=21, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=cfc44517919312ea044ca4a33874e9d9, ASSIGN in 566 msec 2023-07-16 21:15:41,626 INFO [StoreOpener-a0a662f8b472889e1ea9acbed9048152-1] regionserver.HStore(310): Store=a0a662f8b472889e1ea9acbed9048152/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:41,627 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-07-16 21:15:41,627 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure 2dfe6a59994601d58418deb273b19667, server=jenkins-hbase4.apache.org,36993,1689542134044 in 397 msec 2023-07-16 21:15:41,627 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=e442c5f5ae5f06c0a3bdc2679964b13a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:41,627 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00\\xA2\\xA4\\xA6\\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141627"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542141627"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542141627"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542141627"}]},"ts":"1689542141627"} 2023-07-16 21:15:41,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/a0a662f8b472889e1ea9acbed9048152 2023-07-16 21:15:41,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/a0a662f8b472889e1ea9acbed9048152 2023-07-16 21:15:41,630 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=21, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=2dfe6a59994601d58418deb273b19667, ASSIGN in 579 msec 2023-07-16 21:15:41,630 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=26 2023-07-16 21:15:41,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=26, state=SUCCESS; OpenRegionProcedure 376580fd1a9ca2e27b13627706cff755, server=jenkins-hbase4.apache.org,33511,1689542134438 in 401 msec 2023-07-16 21:15:41,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c240c5e2b0a66ca9d8b1dd1f3d696829 2023-07-16 21:15:41,634 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=21, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=376580fd1a9ca2e27b13627706cff755, ASSIGN in 582 msec 2023-07-16 21:15:41,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a0a662f8b472889e1ea9acbed9048152 2023-07-16 21:15:41,635 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=35, resume processing ppid=28 2023-07-16 21:15:41,635 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=35, ppid=28, state=SUCCESS; OpenRegionProcedure e442c5f5ae5f06c0a3bdc2679964b13a, server=jenkins-hbase4.apache.org,41213,1689542138767 in 415 msec 2023-07-16 21:15:41,638 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=21, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=e442c5f5ae5f06c0a3bdc2679964b13a, ASSIGN in 587 msec 2023-07-16 21:15:41,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/c240c5e2b0a66ca9d8b1dd1f3d696829/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:41,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c240c5e2b0a66ca9d8b1dd1f3d696829; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10337334400, jitterRate=-0.03726071119308472}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:41,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c240c5e2b0a66ca9d8b1dd1f3d696829: 2023-07-16 21:15:41,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/a0a662f8b472889e1ea9acbed9048152/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:41,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a0a662f8b472889e1ea9acbed9048152; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10004986560, jitterRate=-0.06821301579475403}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:41,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a0a662f8b472889e1ea9acbed9048152: 2023-07-16 21:15:41,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829., pid=37, masterSystemTime=1689542141368 2023-07-16 21:15:41,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152., pid=39, masterSystemTime=1689542141377 2023-07-16 21:15:41,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829. 2023-07-16 21:15:41,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829. 2023-07-16 21:15:41,651 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152. 2023-07-16 21:15:41,651 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=27 updating hbase:meta row=c240c5e2b0a66ca9d8b1dd1f3d696829, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:41,651 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00\\x82\\x84\\x86\\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141650"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542141650"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542141650"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542141650"}]},"ts":"1689542141650"} 2023-07-16 21:15:41,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152. 2023-07-16 21:15:41,651 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=a0a662f8b472889e1ea9acbed9048152, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:41,652 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00\\xC2\\xC4\\xC6\\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141651"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542141651"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542141651"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542141651"}]},"ts":"1689542141651"} 2023-07-16 21:15:41,662 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=27 2023-07-16 21:15:41,663 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=27, state=SUCCESS; OpenRegionProcedure c240c5e2b0a66ca9d8b1dd1f3d696829, server=jenkins-hbase4.apache.org,36993,1689542134044 in 434 msec 2023-07-16 21:15:41,664 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=29 2023-07-16 21:15:41,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=29, state=SUCCESS; OpenRegionProcedure a0a662f8b472889e1ea9acbed9048152, server=jenkins-hbase4.apache.org,33511,1689542134438 in 437 msec 2023-07-16 21:15:41,665 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=21, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=c240c5e2b0a66ca9d8b1dd1f3d696829, ASSIGN in 615 msec 2023-07-16 21:15:41,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=21 2023-07-16 21:15:41,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=21, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=a0a662f8b472889e1ea9acbed9048152, ASSIGN in 617 msec 2023-07-16 21:15:41,670 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=21, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=Group_testCreateMultiRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-16 21:15:41,670 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateMultiRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542141670"}]},"ts":"1689542141670"} 2023-07-16 21:15:41,673 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateMultiRegion, state=ENABLED in hbase:meta 2023-07-16 21:15:41,677 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=21, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=Group_testCreateMultiRegion execute state=CREATE_TABLE_POST_OPERATION 2023-07-16 21:15:41,687 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=21, state=SUCCESS; CreateTableProcedure table=Group_testCreateMultiRegion in 908 msec 2023-07-16 21:15:41,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=21 2023-07-16 21:15:41,902 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:Group_testCreateMultiRegion, procId: 21 completed 2023-07-16 21:15:41,902 DEBUG [Listener at localhost/43605] hbase.HBaseTestingUtility(3430): Waiting until all regions of table Group_testCreateMultiRegion get assigned. Timeout = 60000ms 2023-07-16 21:15:41,903 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:41,909 INFO [Listener at localhost/43605] hbase.HBaseTestingUtility(3484): All regions for table Group_testCreateMultiRegion assigned to meta. Checking AM states. 2023-07-16 21:15:41,909 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:41,909 INFO [Listener at localhost/43605] hbase.HBaseTestingUtility(3504): All regions for table Group_testCreateMultiRegion assigned. 2023-07-16 21:15:41,911 INFO [Listener at localhost/43605] client.HBaseAdmin$15(890): Started disable of Group_testCreateMultiRegion 2023-07-16 21:15:41,912 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable Group_testCreateMultiRegion 2023-07-16 21:15:41,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=Group_testCreateMultiRegion 2023-07-16 21:15:41,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=42 2023-07-16 21:15:41,917 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateMultiRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542141917"}]},"ts":"1689542141917"} 2023-07-16 21:15:41,918 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateMultiRegion, state=DISABLING in hbase:meta 2023-07-16 21:15:41,920 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set Group_testCreateMultiRegion to state=DISABLING 2023-07-16 21:15:41,924 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=c7916c21d8a3ef54f6d9fe15de8c3521, UNASSIGN}, {pid=44, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=bc805df06d6faa31e4b1fe04f9136e24, UNASSIGN}, {pid=45, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=e020fe0e78fed6125375355fc8442e1e, UNASSIGN}, {pid=46, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=376580fd1a9ca2e27b13627706cff755, UNASSIGN}, {pid=47, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=c240c5e2b0a66ca9d8b1dd1f3d696829, UNASSIGN}, {pid=48, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=e442c5f5ae5f06c0a3bdc2679964b13a, UNASSIGN}, {pid=49, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=a0a662f8b472889e1ea9acbed9048152, UNASSIGN}, {pid=50, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=cfc44517919312ea044ca4a33874e9d9, UNASSIGN}, {pid=51, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=2dfe6a59994601d58418deb273b19667, UNASSIGN}, {pid=52, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=3a8b97f1d6bb14f6e3b05cb9b89abdf4, UNASSIGN}] 2023-07-16 21:15:41,927 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=bc805df06d6faa31e4b1fe04f9136e24, UNASSIGN 2023-07-16 21:15:41,927 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=43, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=c7916c21d8a3ef54f6d9fe15de8c3521, UNASSIGN 2023-07-16 21:15:41,927 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=e020fe0e78fed6125375355fc8442e1e, UNASSIGN 2023-07-16 21:15:41,928 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=46, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=376580fd1a9ca2e27b13627706cff755, UNASSIGN 2023-07-16 21:15:41,928 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=52, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=3a8b97f1d6bb14f6e3b05cb9b89abdf4, UNASSIGN 2023-07-16 21:15:41,929 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=44 updating hbase:meta row=bc805df06d6faa31e4b1fe04f9136e24, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:41,929 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=e020fe0e78fed6125375355fc8442e1e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:41,929 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=43 updating hbase:meta row=c7916c21d8a3ef54f6d9fe15de8c3521, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:41,929 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141929"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141929"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141929"}]},"ts":"1689542141929"} 2023-07-16 21:15:41,929 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=46 updating hbase:meta row=376580fd1a9ca2e27b13627706cff755, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:41,929 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\"$\u0026(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141929"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141929"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141929"}]},"ts":"1689542141929"} 2023-07-16 21:15:41,929 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141929"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141929"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141929"}]},"ts":"1689542141929"} 2023-07-16 21:15:41,929 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\x02\\x04\\x06\\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141929"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141929"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141929"}]},"ts":"1689542141929"} 2023-07-16 21:15:41,929 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=52 updating hbase:meta row=3a8b97f1d6bb14f6e3b05cb9b89abdf4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:41,930 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689542141929"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141929"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141929"}]},"ts":"1689542141929"} 2023-07-16 21:15:41,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=45, state=RUNNABLE; CloseRegionProcedure e020fe0e78fed6125375355fc8442e1e, server=jenkins-hbase4.apache.org,45229,1689542134233}] 2023-07-16 21:15:41,932 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=44, state=RUNNABLE; CloseRegionProcedure bc805df06d6faa31e4b1fe04f9136e24, server=jenkins-hbase4.apache.org,33511,1689542134438}] 2023-07-16 21:15:41,933 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=46, state=RUNNABLE; CloseRegionProcedure 376580fd1a9ca2e27b13627706cff755, server=jenkins-hbase4.apache.org,33511,1689542134438}] 2023-07-16 21:15:41,935 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=43, state=RUNNABLE; CloseRegionProcedure c7916c21d8a3ef54f6d9fe15de8c3521, server=jenkins-hbase4.apache.org,41213,1689542138767}] 2023-07-16 21:15:41,936 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=2dfe6a59994601d58418deb273b19667, UNASSIGN 2023-07-16 21:15:41,937 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=52, state=RUNNABLE; CloseRegionProcedure 3a8b97f1d6bb14f6e3b05cb9b89abdf4, server=jenkins-hbase4.apache.org,36993,1689542134044}] 2023-07-16 21:15:41,937 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=2dfe6a59994601d58418deb273b19667, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:41,938 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x01\\x03\\x05\\x07\\x09,1689542140769.2dfe6a59994601d58418deb273b19667.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689542141937"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141937"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141937"}]},"ts":"1689542141937"} 2023-07-16 21:15:41,938 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=cfc44517919312ea044ca4a33874e9d9, UNASSIGN 2023-07-16 21:15:41,939 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=49, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=a0a662f8b472889e1ea9acbed9048152, UNASSIGN 2023-07-16 21:15:41,939 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=51, state=RUNNABLE; CloseRegionProcedure 2dfe6a59994601d58418deb273b19667, server=jenkins-hbase4.apache.org,36993,1689542134044}] 2023-07-16 21:15:41,940 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=cfc44517919312ea044ca4a33874e9d9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:41,940 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\xE2\\xE4\\xE6\\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141940"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141940"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141940"}]},"ts":"1689542141940"} 2023-07-16 21:15:41,941 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=49 updating hbase:meta row=a0a662f8b472889e1ea9acbed9048152, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:41,941 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\xC2\\xC4\\xC6\\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141941"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141941"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141941"}]},"ts":"1689542141941"} 2023-07-16 21:15:41,942 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=48, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=e442c5f5ae5f06c0a3bdc2679964b13a, UNASSIGN 2023-07-16 21:15:41,942 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, ppid=42, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=c240c5e2b0a66ca9d8b1dd1f3d696829, UNASSIGN 2023-07-16 21:15:41,942 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=50, state=RUNNABLE; CloseRegionProcedure cfc44517919312ea044ca4a33874e9d9, server=jenkins-hbase4.apache.org,45229,1689542134233}] 2023-07-16 21:15:41,944 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=48 updating hbase:meta row=e442c5f5ae5f06c0a3bdc2679964b13a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:41,944 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=c240c5e2b0a66ca9d8b1dd1f3d696829, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:41,944 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\xA2\\xA4\\xA6\\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141944"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141944"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141944"}]},"ts":"1689542141944"} 2023-07-16 21:15:41,944 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\x82\\x84\\x86\\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542141944"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542141944"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542141944"}]},"ts":"1689542141944"} 2023-07-16 21:15:41,944 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=49, state=RUNNABLE; CloseRegionProcedure a0a662f8b472889e1ea9acbed9048152, server=jenkins-hbase4.apache.org,33511,1689542134438}] 2023-07-16 21:15:41,947 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=48, state=RUNNABLE; CloseRegionProcedure e442c5f5ae5f06c0a3bdc2679964b13a, server=jenkins-hbase4.apache.org,41213,1689542138767}] 2023-07-16 21:15:41,948 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=47, state=RUNNABLE; CloseRegionProcedure c240c5e2b0a66ca9d8b1dd1f3d696829, server=jenkins-hbase4.apache.org,36993,1689542134044}] 2023-07-16 21:15:42,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=42 2023-07-16 21:15:42,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close cfc44517919312ea044ca4a33874e9d9 2023-07-16 21:15:42,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing cfc44517919312ea044ca4a33874e9d9, disabling compactions & flushes 2023-07-16 21:15:42,085 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9. 2023-07-16 21:15:42,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9. 2023-07-16 21:15:42,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9. after waiting 0 ms 2023-07-16 21:15:42,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9. 2023-07-16 21:15:42,086 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 376580fd1a9ca2e27b13627706cff755 2023-07-16 21:15:42,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 376580fd1a9ca2e27b13627706cff755, disabling compactions & flushes 2023-07-16 21:15:42,087 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755. 2023-07-16 21:15:42,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755. 2023-07-16 21:15:42,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755. after waiting 0 ms 2023-07-16 21:15:42,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755. 2023-07-16 21:15:42,091 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close e442c5f5ae5f06c0a3bdc2679964b13a 2023-07-16 21:15:42,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e442c5f5ae5f06c0a3bdc2679964b13a, disabling compactions & flushes 2023-07-16 21:15:42,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a. 2023-07-16 21:15:42,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a. 2023-07-16 21:15:42,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a. after waiting 0 ms 2023-07-16 21:15:42,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a. 2023-07-16 21:15:42,095 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c240c5e2b0a66ca9d8b1dd1f3d696829 2023-07-16 21:15:42,098 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/cfc44517919312ea044ca4a33874e9d9/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:15:42,098 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/376580fd1a9ca2e27b13627706cff755/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:15:42,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c240c5e2b0a66ca9d8b1dd1f3d696829, disabling compactions & flushes 2023-07-16 21:15:42,099 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829. 2023-07-16 21:15:42,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829. 2023-07-16 21:15:42,100 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9. 2023-07-16 21:15:42,100 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829. after waiting 0 ms 2023-07-16 21:15:42,100 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for cfc44517919312ea044ca4a33874e9d9: 2023-07-16 21:15:42,100 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829. 2023-07-16 21:15:42,101 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755. 2023-07-16 21:15:42,101 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 376580fd1a9ca2e27b13627706cff755: 2023-07-16 21:15:42,103 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed cfc44517919312ea044ca4a33874e9d9 2023-07-16 21:15:42,103 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close e020fe0e78fed6125375355fc8442e1e 2023-07-16 21:15:42,104 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e020fe0e78fed6125375355fc8442e1e, disabling compactions & flushes 2023-07-16 21:15:42,104 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e. 2023-07-16 21:15:42,104 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e. 2023-07-16 21:15:42,104 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e. after waiting 0 ms 2023-07-16 21:15:42,104 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e. 2023-07-16 21:15:42,112 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=cfc44517919312ea044ca4a33874e9d9, regionState=CLOSED 2023-07-16 21:15:42,112 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\xE2\\xE4\\xE6\\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542142112"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542142112"}]},"ts":"1689542142112"} 2023-07-16 21:15:42,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/e442c5f5ae5f06c0a3bdc2679964b13a/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:15:42,113 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 376580fd1a9ca2e27b13627706cff755 2023-07-16 21:15:42,113 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close a0a662f8b472889e1ea9acbed9048152 2023-07-16 21:15:42,114 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=46 updating hbase:meta row=376580fd1a9ca2e27b13627706cff755, regionState=CLOSED 2023-07-16 21:15:42,114 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a0a662f8b472889e1ea9acbed9048152, disabling compactions & flushes 2023-07-16 21:15:42,114 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542142113"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542142113"}]},"ts":"1689542142113"} 2023-07-16 21:15:42,114 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a. 2023-07-16 21:15:42,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e442c5f5ae5f06c0a3bdc2679964b13a: 2023-07-16 21:15:42,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152. 2023-07-16 21:15:42,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152. 2023-07-16 21:15:42,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152. after waiting 0 ms 2023-07-16 21:15:42,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152. 2023-07-16 21:15:42,117 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed e442c5f5ae5f06c0a3bdc2679964b13a 2023-07-16 21:15:42,117 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c7916c21d8a3ef54f6d9fe15de8c3521 2023-07-16 21:15:42,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c7916c21d8a3ef54f6d9fe15de8c3521, disabling compactions & flushes 2023-07-16 21:15:42,119 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521. 2023-07-16 21:15:42,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521. 2023-07-16 21:15:42,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521. after waiting 0 ms 2023-07-16 21:15:42,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521. 2023-07-16 21:15:42,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/e020fe0e78fed6125375355fc8442e1e/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:15:42,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/c240c5e2b0a66ca9d8b1dd1f3d696829/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:15:42,120 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829. 2023-07-16 21:15:42,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c240c5e2b0a66ca9d8b1dd1f3d696829: 2023-07-16 21:15:42,123 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=48 updating hbase:meta row=e442c5f5ae5f06c0a3bdc2679964b13a, regionState=CLOSED 2023-07-16 21:15:42,123 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\xA2\\xA4\\xA6\\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542142122"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542142122"}]},"ts":"1689542142122"} 2023-07-16 21:15:42,124 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=50 2023-07-16 21:15:42,124 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=50, state=SUCCESS; CloseRegionProcedure cfc44517919312ea044ca4a33874e9d9, server=jenkins-hbase4.apache.org,45229,1689542134233 in 174 msec 2023-07-16 21:15:42,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e. 2023-07-16 21:15:42,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e020fe0e78fed6125375355fc8442e1e: 2023-07-16 21:15:42,127 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c240c5e2b0a66ca9d8b1dd1f3d696829 2023-07-16 21:15:42,127 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 3a8b97f1d6bb14f6e3b05cb9b89abdf4 2023-07-16 21:15:42,127 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=46 2023-07-16 21:15:42,127 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=46, state=SUCCESS; CloseRegionProcedure 376580fd1a9ca2e27b13627706cff755, server=jenkins-hbase4.apache.org,33511,1689542134438 in 184 msec 2023-07-16 21:15:42,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3a8b97f1d6bb14f6e3b05cb9b89abdf4, disabling compactions & flushes 2023-07-16 21:15:42,129 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4. 2023-07-16 21:15:42,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4. 2023-07-16 21:15:42,129 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=c240c5e2b0a66ca9d8b1dd1f3d696829, regionState=CLOSED 2023-07-16 21:15:42,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4. after waiting 0 ms 2023-07-16 21:15:42,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4. 2023-07-16 21:15:42,129 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\x82\\x84\\x86\\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542142129"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542142129"}]},"ts":"1689542142129"} 2023-07-16 21:15:42,134 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=42, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=cfc44517919312ea044ca4a33874e9d9, UNASSIGN in 200 msec 2023-07-16 21:15:42,135 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed e020fe0e78fed6125375355fc8442e1e 2023-07-16 21:15:42,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/c7916c21d8a3ef54f6d9fe15de8c3521/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:15:42,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/a0a662f8b472889e1ea9acbed9048152/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:15:42,138 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521. 2023-07-16 21:15:42,138 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152. 2023-07-16 21:15:42,138 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c7916c21d8a3ef54f6d9fe15de8c3521: 2023-07-16 21:15:42,138 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=e020fe0e78fed6125375355fc8442e1e, regionState=CLOSED 2023-07-16 21:15:42,138 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=42, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=376580fd1a9ca2e27b13627706cff755, UNASSIGN in 206 msec 2023-07-16 21:15:42,138 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a0a662f8b472889e1ea9acbed9048152: 2023-07-16 21:15:42,140 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542142138"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542142138"}]},"ts":"1689542142138"} 2023-07-16 21:15:42,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=48 2023-07-16 21:15:42,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=48, state=SUCCESS; CloseRegionProcedure e442c5f5ae5f06c0a3bdc2679964b13a, server=jenkins-hbase4.apache.org,41213,1689542138767 in 186 msec 2023-07-16 21:15:42,144 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=47 2023-07-16 21:15:42,145 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=47, state=SUCCESS; CloseRegionProcedure c240c5e2b0a66ca9d8b1dd1f3d696829, server=jenkins-hbase4.apache.org,36993,1689542134044 in 187 msec 2023-07-16 21:15:42,145 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c7916c21d8a3ef54f6d9fe15de8c3521 2023-07-16 21:15:42,146 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/3a8b97f1d6bb14f6e3b05cb9b89abdf4/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:15:42,147 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4. 2023-07-16 21:15:42,147 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3a8b97f1d6bb14f6e3b05cb9b89abdf4: 2023-07-16 21:15:42,150 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=43 updating hbase:meta row=c7916c21d8a3ef54f6d9fe15de8c3521, regionState=CLOSED 2023-07-16 21:15:42,150 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\x02\\x04\\x06\\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542142150"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542142150"}]},"ts":"1689542142150"} 2023-07-16 21:15:42,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed a0a662f8b472889e1ea9acbed9048152 2023-07-16 21:15:42,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close bc805df06d6faa31e4b1fe04f9136e24 2023-07-16 21:15:42,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bc805df06d6faa31e4b1fe04f9136e24, disabling compactions & flushes 2023-07-16 21:15:42,153 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24. 2023-07-16 21:15:42,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24. 2023-07-16 21:15:42,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24. after waiting 0 ms 2023-07-16 21:15:42,153 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=42, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=e442c5f5ae5f06c0a3bdc2679964b13a, UNASSIGN in 219 msec 2023-07-16 21:15:42,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24. 2023-07-16 21:15:42,153 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=49 updating hbase:meta row=a0a662f8b472889e1ea9acbed9048152, regionState=CLOSED 2023-07-16 21:15:42,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=42, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=c240c5e2b0a66ca9d8b1dd1f3d696829, UNASSIGN in 224 msec 2023-07-16 21:15:42,154 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\xC2\\xC4\\xC6\\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542142153"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542142153"}]},"ts":"1689542142153"} 2023-07-16 21:15:42,155 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 3a8b97f1d6bb14f6e3b05cb9b89abdf4 2023-07-16 21:15:42,155 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 2dfe6a59994601d58418deb273b19667 2023-07-16 21:15:42,156 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2dfe6a59994601d58418deb273b19667, disabling compactions & flushes 2023-07-16 21:15:42,156 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667. 2023-07-16 21:15:42,156 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667. 2023-07-16 21:15:42,156 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=52 updating hbase:meta row=3a8b97f1d6bb14f6e3b05cb9b89abdf4, regionState=CLOSED 2023-07-16 21:15:42,156 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667. after waiting 0 ms 2023-07-16 21:15:42,156 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667. 2023-07-16 21:15:42,156 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689542142156"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542142156"}]},"ts":"1689542142156"} 2023-07-16 21:15:42,158 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=45 2023-07-16 21:15:42,158 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=45, state=SUCCESS; CloseRegionProcedure e020fe0e78fed6125375355fc8442e1e, server=jenkins-hbase4.apache.org,45229,1689542134233 in 220 msec 2023-07-16 21:15:42,162 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=43 2023-07-16 21:15:42,162 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=43, state=SUCCESS; CloseRegionProcedure c7916c21d8a3ef54f6d9fe15de8c3521, server=jenkins-hbase4.apache.org,41213,1689542138767 in 222 msec 2023-07-16 21:15:42,163 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=42, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=e020fe0e78fed6125375355fc8442e1e, UNASSIGN in 237 msec 2023-07-16 21:15:42,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/bc805df06d6faa31e4b1fe04f9136e24/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:15:42,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateMultiRegion/2dfe6a59994601d58418deb273b19667/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:15:42,165 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24. 2023-07-16 21:15:42,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bc805df06d6faa31e4b1fe04f9136e24: 2023-07-16 21:15:42,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667. 2023-07-16 21:15:42,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2dfe6a59994601d58418deb273b19667: 2023-07-16 21:15:42,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=c7916c21d8a3ef54f6d9fe15de8c3521, UNASSIGN in 241 msec 2023-07-16 21:15:42,166 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=49 2023-07-16 21:15:42,167 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=49, state=SUCCESS; CloseRegionProcedure a0a662f8b472889e1ea9acbed9048152, server=jenkins-hbase4.apache.org,33511,1689542134438 in 216 msec 2023-07-16 21:15:42,168 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=52 2023-07-16 21:15:42,169 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=52, state=SUCCESS; CloseRegionProcedure 3a8b97f1d6bb14f6e3b05cb9b89abdf4, server=jenkins-hbase4.apache.org,36993,1689542134044 in 226 msec 2023-07-16 21:15:42,169 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed bc805df06d6faa31e4b1fe04f9136e24 2023-07-16 21:15:42,170 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=49, ppid=42, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=a0a662f8b472889e1ea9acbed9048152, UNASSIGN in 243 msec 2023-07-16 21:15:42,170 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=44 updating hbase:meta row=bc805df06d6faa31e4b1fe04f9136e24, regionState=CLOSED 2023-07-16 21:15:42,170 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 2dfe6a59994601d58418deb273b19667 2023-07-16 21:15:42,170 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\"$\u0026(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542142170"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542142170"}]},"ts":"1689542142170"} 2023-07-16 21:15:42,173 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=42, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=3a8b97f1d6bb14f6e3b05cb9b89abdf4, UNASSIGN in 245 msec 2023-07-16 21:15:42,173 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=2dfe6a59994601d58418deb273b19667, regionState=CLOSED 2023-07-16 21:15:42,173 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x01\\x03\\x05\\x07\\x09,1689542140769.2dfe6a59994601d58418deb273b19667.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689542142173"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542142173"}]},"ts":"1689542142173"} 2023-07-16 21:15:42,189 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=44 2023-07-16 21:15:42,189 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=44, state=SUCCESS; CloseRegionProcedure bc805df06d6faa31e4b1fe04f9136e24, server=jenkins-hbase4.apache.org,33511,1689542134438 in 242 msec 2023-07-16 21:15:42,190 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=51 2023-07-16 21:15:42,191 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=51, state=SUCCESS; CloseRegionProcedure 2dfe6a59994601d58418deb273b19667, server=jenkins-hbase4.apache.org,36993,1689542134044 in 247 msec 2023-07-16 21:15:42,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=42, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=bc805df06d6faa31e4b1fe04f9136e24, UNASSIGN in 268 msec 2023-07-16 21:15:42,193 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=42 2023-07-16 21:15:42,193 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=42, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=2dfe6a59994601d58418deb273b19667, UNASSIGN in 267 msec 2023-07-16 21:15:42,194 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateMultiRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542142194"}]},"ts":"1689542142194"} 2023-07-16 21:15:42,196 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateMultiRegion, state=DISABLED in hbase:meta 2023-07-16 21:15:42,198 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set Group_testCreateMultiRegion to state=DISABLED 2023-07-16 21:15:42,201 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; DisableTableProcedure table=Group_testCreateMultiRegion in 287 msec 2023-07-16 21:15:42,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=42 2023-07-16 21:15:42,219 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:Group_testCreateMultiRegion, procId: 42 completed 2023-07-16 21:15:42,220 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete Group_testCreateMultiRegion 2023-07-16 21:15:42,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-16 21:15:42,224 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=63, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-16 21:15:42,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint(577): Removing deleted table 'Group_testCreateMultiRegion' from rsgroup 'default' 2023-07-16 21:15:42,227 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=63, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-16 21:15:42,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:42,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:42,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:42,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-07-16 21:15:42,246 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c7916c21d8a3ef54f6d9fe15de8c3521 2023-07-16 21:15:42,246 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e020fe0e78fed6125375355fc8442e1e 2023-07-16 21:15:42,246 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/bc805df06d6faa31e4b1fe04f9136e24 2023-07-16 21:15:42,247 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/376580fd1a9ca2e27b13627706cff755 2023-07-16 21:15:42,247 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c240c5e2b0a66ca9d8b1dd1f3d696829 2023-07-16 21:15:42,247 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e442c5f5ae5f06c0a3bdc2679964b13a 2023-07-16 21:15:42,247 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/a0a662f8b472889e1ea9acbed9048152 2023-07-16 21:15:42,247 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/cfc44517919312ea044ca4a33874e9d9 2023-07-16 21:15:42,252 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/376580fd1a9ca2e27b13627706cff755/f, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/376580fd1a9ca2e27b13627706cff755/recovered.edits] 2023-07-16 21:15:42,253 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c7916c21d8a3ef54f6d9fe15de8c3521/f, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c7916c21d8a3ef54f6d9fe15de8c3521/recovered.edits] 2023-07-16 21:15:42,253 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e020fe0e78fed6125375355fc8442e1e/f, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e020fe0e78fed6125375355fc8442e1e/recovered.edits] 2023-07-16 21:15:42,254 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/a0a662f8b472889e1ea9acbed9048152/f, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/a0a662f8b472889e1ea9acbed9048152/recovered.edits] 2023-07-16 21:15:42,254 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e442c5f5ae5f06c0a3bdc2679964b13a/f, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e442c5f5ae5f06c0a3bdc2679964b13a/recovered.edits] 2023-07-16 21:15:42,258 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/cfc44517919312ea044ca4a33874e9d9/f, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/cfc44517919312ea044ca4a33874e9d9/recovered.edits] 2023-07-16 21:15:42,258 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/bc805df06d6faa31e4b1fe04f9136e24/f, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/bc805df06d6faa31e4b1fe04f9136e24/recovered.edits] 2023-07-16 21:15:42,258 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c240c5e2b0a66ca9d8b1dd1f3d696829/f, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c240c5e2b0a66ca9d8b1dd1f3d696829/recovered.edits] 2023-07-16 21:15:42,278 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/376580fd1a9ca2e27b13627706cff755/recovered.edits/4.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/default/Group_testCreateMultiRegion/376580fd1a9ca2e27b13627706cff755/recovered.edits/4.seqid 2023-07-16 21:15:42,279 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c7916c21d8a3ef54f6d9fe15de8c3521/recovered.edits/4.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/default/Group_testCreateMultiRegion/c7916c21d8a3ef54f6d9fe15de8c3521/recovered.edits/4.seqid 2023-07-16 21:15:42,280 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/376580fd1a9ca2e27b13627706cff755 2023-07-16 21:15:42,280 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/2dfe6a59994601d58418deb273b19667 2023-07-16 21:15:42,280 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e442c5f5ae5f06c0a3bdc2679964b13a/recovered.edits/4.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/default/Group_testCreateMultiRegion/e442c5f5ae5f06c0a3bdc2679964b13a/recovered.edits/4.seqid 2023-07-16 21:15:42,281 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e020fe0e78fed6125375355fc8442e1e/recovered.edits/4.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/default/Group_testCreateMultiRegion/e020fe0e78fed6125375355fc8442e1e/recovered.edits/4.seqid 2023-07-16 21:15:42,282 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/cfc44517919312ea044ca4a33874e9d9/recovered.edits/4.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/default/Group_testCreateMultiRegion/cfc44517919312ea044ca4a33874e9d9/recovered.edits/4.seqid 2023-07-16 21:15:42,282 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e442c5f5ae5f06c0a3bdc2679964b13a 2023-07-16 21:15:42,282 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/3a8b97f1d6bb14f6e3b05cb9b89abdf4 2023-07-16 21:15:42,284 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/bc805df06d6faa31e4b1fe04f9136e24/recovered.edits/4.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/default/Group_testCreateMultiRegion/bc805df06d6faa31e4b1fe04f9136e24/recovered.edits/4.seqid 2023-07-16 21:15:42,285 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/cfc44517919312ea044ca4a33874e9d9 2023-07-16 21:15:42,285 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c7916c21d8a3ef54f6d9fe15de8c3521 2023-07-16 21:15:42,285 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/e020fe0e78fed6125375355fc8442e1e 2023-07-16 21:15:42,286 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c240c5e2b0a66ca9d8b1dd1f3d696829/recovered.edits/4.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/default/Group_testCreateMultiRegion/c240c5e2b0a66ca9d8b1dd1f3d696829/recovered.edits/4.seqid 2023-07-16 21:15:42,286 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/bc805df06d6faa31e4b1fe04f9136e24 2023-07-16 21:15:42,287 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/a0a662f8b472889e1ea9acbed9048152/recovered.edits/4.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/default/Group_testCreateMultiRegion/a0a662f8b472889e1ea9acbed9048152/recovered.edits/4.seqid 2023-07-16 21:15:42,287 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/c240c5e2b0a66ca9d8b1dd1f3d696829 2023-07-16 21:15:42,287 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/2dfe6a59994601d58418deb273b19667/f, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/2dfe6a59994601d58418deb273b19667/recovered.edits] 2023-07-16 21:15:42,288 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/3a8b97f1d6bb14f6e3b05cb9b89abdf4/f, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/3a8b97f1d6bb14f6e3b05cb9b89abdf4/recovered.edits] 2023-07-16 21:15:42,288 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/a0a662f8b472889e1ea9acbed9048152 2023-07-16 21:15:42,300 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/2dfe6a59994601d58418deb273b19667/recovered.edits/4.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/default/Group_testCreateMultiRegion/2dfe6a59994601d58418deb273b19667/recovered.edits/4.seqid 2023-07-16 21:15:42,301 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/3a8b97f1d6bb14f6e3b05cb9b89abdf4/recovered.edits/4.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/default/Group_testCreateMultiRegion/3a8b97f1d6bb14f6e3b05cb9b89abdf4/recovered.edits/4.seqid 2023-07-16 21:15:42,301 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/2dfe6a59994601d58418deb273b19667 2023-07-16 21:15:42,301 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateMultiRegion/3a8b97f1d6bb14f6e3b05cb9b89abdf4 2023-07-16 21:15:42,302 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived Group_testCreateMultiRegion regions 2023-07-16 21:15:42,305 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=63, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-16 21:15:42,310 WARN [PEWorker-4] procedure.DeleteTableProcedure(384): Deleting some vestigial 10 rows of Group_testCreateMultiRegion from hbase:meta 2023-07-16 21:15:42,313 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(421): Removing 'Group_testCreateMultiRegion' descriptor. 2023-07-16 21:15:42,315 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=63, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-16 21:15:42,315 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(411): Removing 'Group_testCreateMultiRegion' from region states. 2023-07-16 21:15:42,315 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00\\x02\\x04\\x06\\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542142315"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:42,315 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00\"$\u0026(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542142315"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:42,315 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542142315"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:42,316 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542142315"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:42,316 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00\\x82\\x84\\x86\\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542142315"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:42,316 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00\\xA2\\xA4\\xA6\\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542142315"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:42,316 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00\\xC2\\xC4\\xC6\\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542142315"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:42,316 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00\\xE2\\xE4\\xE6\\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542142315"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:42,316 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x01\\x03\\x05\\x07\\x09,1689542140769.2dfe6a59994601d58418deb273b19667.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542142315"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:42,316 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542142315"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:42,319 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 10 regions from META 2023-07-16 21:15:42,319 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => c7916c21d8a3ef54f6d9fe15de8c3521, NAME => 'Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1689542140769.c7916c21d8a3ef54f6d9fe15de8c3521.', STARTKEY => '\x00\x02\x04\x06\x08', ENDKEY => '\x00"$&('}, {ENCODED => bc805df06d6faa31e4b1fe04f9136e24, NAME => 'Group_testCreateMultiRegion,\x00"$&(,1689542140769.bc805df06d6faa31e4b1fe04f9136e24.', STARTKEY => '\x00"$&(', ENDKEY => '\x00BDFH'}, {ENCODED => e020fe0e78fed6125375355fc8442e1e, NAME => 'Group_testCreateMultiRegion,\x00BDFH,1689542140769.e020fe0e78fed6125375355fc8442e1e.', STARTKEY => '\x00BDFH', ENDKEY => '\x00bdfh'}, {ENCODED => 376580fd1a9ca2e27b13627706cff755, NAME => 'Group_testCreateMultiRegion,\x00bdfh,1689542140769.376580fd1a9ca2e27b13627706cff755.', STARTKEY => '\x00bdfh', ENDKEY => '\x00\x82\x84\x86\x88'}, {ENCODED => c240c5e2b0a66ca9d8b1dd1f3d696829, NAME => 'Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1689542140769.c240c5e2b0a66ca9d8b1dd1f3d696829.', STARTKEY => '\x00\x82\x84\x86\x88', ENDKEY => '\x00\xA2\xA4\xA6\xA8'}, {ENCODED => e442c5f5ae5f06c0a3bdc2679964b13a, NAME => 'Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1689542140769.e442c5f5ae5f06c0a3bdc2679964b13a.', STARTKEY => '\x00\xA2\xA4\xA6\xA8', ENDKEY => '\x00\xC2\xC4\xC6\xC8'}, {ENCODED => a0a662f8b472889e1ea9acbed9048152, NAME => 'Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1689542140769.a0a662f8b472889e1ea9acbed9048152.', STARTKEY => '\x00\xC2\xC4\xC6\xC8', ENDKEY => '\x00\xE2\xE4\xE6\xE8'}, {ENCODED => cfc44517919312ea044ca4a33874e9d9, NAME => 'Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1689542140769.cfc44517919312ea044ca4a33874e9d9.', STARTKEY => '\x00\xE2\xE4\xE6\xE8', ENDKEY => '\x01\x03\x05\x07\x09'}, {ENCODED => 2dfe6a59994601d58418deb273b19667, NAME => 'Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1689542140769.2dfe6a59994601d58418deb273b19667.', STARTKEY => '\x01\x03\x05\x07\x09', ENDKEY => ''}, {ENCODED => 3a8b97f1d6bb14f6e3b05cb9b89abdf4, NAME => 'Group_testCreateMultiRegion,,1689542140769.3a8b97f1d6bb14f6e3b05cb9b89abdf4.', STARTKEY => '', ENDKEY => '\x00\x02\x04\x06\x08'}] 2023-07-16 21:15:42,319 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(415): Marking 'Group_testCreateMultiRegion' as deleted. 2023-07-16 21:15:42,319 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689542142319"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:42,321 INFO [PEWorker-4] hbase.MetaTableAccessor(1658): Deleted table Group_testCreateMultiRegion state from META 2023-07-16 21:15:42,324 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(130): Finished pid=63, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-16 21:15:42,326 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DeleteTableProcedure table=Group_testCreateMultiRegion in 104 msec 2023-07-16 21:15:42,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-07-16 21:15:42,340 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:Group_testCreateMultiRegion, procId: 63 completed 2023-07-16 21:15:42,345 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:42,346 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:42,348 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:42,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:42,348 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:42,350 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:42,350 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:42,351 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:42,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:42,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:42,359 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:42,364 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:15:42,365 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:42,367 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-16 21:15:42,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:42,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:42,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:42,371 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:42,375 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:42,375 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:42,378 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:42,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:42,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.CallRunner(144): callId: 246 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:39404 deadline: 1689543342378, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:42,379 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:42,382 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:42,383 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:42,384 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:42,384 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:42,385 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:42,385 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:42,410 INFO [Listener at localhost/43605] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testCreateMultiRegion Thread=503 (was 497) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-870215550_17 at /127.0.0.1:45040 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1114267645_17 at /127.0.0.1:45048 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x50aa0278-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_628029656_17 at /127.0.0.1:55766 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=773 (was 749) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=312 (was 312), ProcessCount=172 (was 172), AvailableMemoryMB=2831 (was 2995) 2023-07-16 21:15:42,410 WARN [Listener at localhost/43605] hbase.ResourceChecker(130): Thread=503 is superior to 500 2023-07-16 21:15:42,431 INFO [Listener at localhost/43605] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testNamespaceCreateAndAssign Thread=503, OpenFileDescriptor=773, MaxFileDescriptor=60000, SystemLoadAverage=312, ProcessCount=172, AvailableMemoryMB=2829 2023-07-16 21:15:42,431 WARN [Listener at localhost/43605] hbase.ResourceChecker(130): Thread=503 is superior to 500 2023-07-16 21:15:42,431 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(132): testNamespaceCreateAndAssign 2023-07-16 21:15:42,437 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:42,437 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:42,439 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:42,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:42,439 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:42,440 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:42,440 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:42,442 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:42,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:42,448 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver 2023-07-16 21:15:42,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:42,448 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver Metrics about HBase MasterObservers 2023-07-16 21:15:42,449 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-16 21:15:42,449 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-16 21:15:42,449 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint 2023-07-16 21:15:42,449 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint Metrics about HBase MasterObservers 2023-07-16 21:15:42,450 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:42,451 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-16 21:15:42,453 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:rsgroup' 2023-07-16 21:15:42,454 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-16 21:15:42,454 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:15:42,455 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:42,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:42,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:42,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:42,463 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:42,466 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:42,466 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:42,469 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:42,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:42,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.CallRunner(144): callId: 274 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:39404 deadline: 1689543342469, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:42,470 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:42,472 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:42,472 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:42,473 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:42,473 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:42,474 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:42,474 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:42,474 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBasics(118): testNamespaceCreateAndAssign 2023-07-16 21:15:42,475 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:42,475 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:42,476 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup appInfo 2023-07-16 21:15:42,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:42,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:42,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-16 21:15:42,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:15:42,487 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:42,490 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:42,490 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:42,493 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:33511] to rsgroup appInfo 2023-07-16 21:15:42,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:42,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:42,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-16 21:15:42,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:15:42,499 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(286): Moving 0 region(s) to group default, current retry=0 2023-07-16 21:15:42,499 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(261): All regions from [jenkins-hbase4.apache.org,33511,1689542134438] are moved back to default 2023-07-16 21:15:42,499 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(438): Move servers done: default => appInfo 2023-07-16 21:15:42,499 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:42,502 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:42,502 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:42,505 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=appInfo 2023-07-16 21:15:42,505 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:42,511 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'Group_foo', hbase.rsgroup.name => 'appInfo'} 2023-07-16 21:15:42,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=Group_foo 2023-07-16 21:15:42,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=64 2023-07-16 21:15:42,521 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-16 21:15:42,524 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; CreateNamespaceProcedure, namespace=Group_foo in 12 msec 2023-07-16 21:15:42,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=64 2023-07-16 21:15:42,621 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'Group_foo:Group_testCreateAndAssign', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-16 21:15:42,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-16 21:15:42,624 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=65, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign execute state=CREATE_TABLE_PRE_OPERATION 2023-07-16 21:15:42,624 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "Group_foo" qualifier: "Group_testCreateAndAssign" procId is: 65 2023-07-16 21:15:42,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=65 2023-07-16 21:15:42,627 DEBUG [PEWorker-1] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:42,628 DEBUG [PEWorker-1] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:42,628 DEBUG [PEWorker-1] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-16 21:15:42,629 DEBUG [PEWorker-1] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:15:42,632 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=65, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-16 21:15:42,634 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_foo/Group_testCreateAndAssign/647cf4e5ff75bcc781bc104a14f56d99 2023-07-16 21:15:42,635 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_foo/Group_testCreateAndAssign/647cf4e5ff75bcc781bc104a14f56d99 empty. 2023-07-16 21:15:42,635 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_foo/Group_testCreateAndAssign/647cf4e5ff75bcc781bc104a14f56d99 2023-07-16 21:15:42,635 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived Group_foo:Group_testCreateAndAssign regions 2023-07-16 21:15:42,653 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_foo/Group_testCreateAndAssign/.tabledesc/.tableinfo.0000000001 2023-07-16 21:15:42,654 INFO [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(7675): creating {ENCODED => 647cf4e5ff75bcc781bc104a14f56d99, NAME => 'Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='Group_foo:Group_testCreateAndAssign', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:42,698 DEBUG [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(866): Instantiated Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:42,698 DEBUG [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(1604): Closing 647cf4e5ff75bcc781bc104a14f56d99, disabling compactions & flushes 2023-07-16 21:15:42,698 INFO [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(1626): Closing region Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99. 2023-07-16 21:15:42,698 DEBUG [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99. 2023-07-16 21:15:42,698 DEBUG [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99. after waiting 0 ms 2023-07-16 21:15:42,698 DEBUG [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99. 2023-07-16 21:15:42,698 INFO [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(1838): Closed Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99. 2023-07-16 21:15:42,698 DEBUG [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(1558): Region close journal for 647cf4e5ff75bcc781bc104a14f56d99: 2023-07-16 21:15:42,710 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=65, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign execute state=CREATE_TABLE_ADD_TO_META 2023-07-16 21:15:42,712 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99.","families":{"info":[{"qualifier":"regioninfo","vlen":61,"tag":[],"timestamp":"1689542142711"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542142711"}]},"ts":"1689542142711"} 2023-07-16 21:15:42,713 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-16 21:15:42,714 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=65, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-16 21:15:42,715 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_foo:Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542142714"}]},"ts":"1689542142714"} 2023-07-16 21:15:42,716 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=Group_foo:Group_testCreateAndAssign, state=ENABLING in hbase:meta 2023-07-16 21:15:42,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_foo:Group_testCreateAndAssign, region=647cf4e5ff75bcc781bc104a14f56d99, ASSIGN}] 2023-07-16 21:15:42,722 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=66, ppid=65, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_foo:Group_testCreateAndAssign, region=647cf4e5ff75bcc781bc104a14f56d99, ASSIGN 2023-07-16 21:15:42,723 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=66, ppid=65, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_foo:Group_testCreateAndAssign, region=647cf4e5ff75bcc781bc104a14f56d99, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33511,1689542134438; forceNewPlan=false, retain=false 2023-07-16 21:15:42,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=65 2023-07-16 21:15:42,875 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=66 updating hbase:meta row=647cf4e5ff75bcc781bc104a14f56d99, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:42,875 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99.","families":{"info":[{"qualifier":"regioninfo","vlen":61,"tag":[],"timestamp":"1689542142875"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542142875"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542142875"}]},"ts":"1689542142875"} 2023-07-16 21:15:42,877 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE; OpenRegionProcedure 647cf4e5ff75bcc781bc104a14f56d99, server=jenkins-hbase4.apache.org,33511,1689542134438}] 2023-07-16 21:15:42,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=65 2023-07-16 21:15:43,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99. 2023-07-16 21:15:43,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 647cf4e5ff75bcc781bc104a14f56d99, NAME => 'Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:15:43,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateAndAssign 647cf4e5ff75bcc781bc104a14f56d99 2023-07-16 21:15:43,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:43,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 647cf4e5ff75bcc781bc104a14f56d99 2023-07-16 21:15:43,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 647cf4e5ff75bcc781bc104a14f56d99 2023-07-16 21:15:43,035 INFO [StoreOpener-647cf4e5ff75bcc781bc104a14f56d99-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region 647cf4e5ff75bcc781bc104a14f56d99 2023-07-16 21:15:43,037 DEBUG [StoreOpener-647cf4e5ff75bcc781bc104a14f56d99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/Group_foo/Group_testCreateAndAssign/647cf4e5ff75bcc781bc104a14f56d99/f 2023-07-16 21:15:43,037 DEBUG [StoreOpener-647cf4e5ff75bcc781bc104a14f56d99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/Group_foo/Group_testCreateAndAssign/647cf4e5ff75bcc781bc104a14f56d99/f 2023-07-16 21:15:43,037 INFO [StoreOpener-647cf4e5ff75bcc781bc104a14f56d99-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 647cf4e5ff75bcc781bc104a14f56d99 columnFamilyName f 2023-07-16 21:15:43,038 INFO [StoreOpener-647cf4e5ff75bcc781bc104a14f56d99-1] regionserver.HStore(310): Store=647cf4e5ff75bcc781bc104a14f56d99/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:43,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/Group_foo/Group_testCreateAndAssign/647cf4e5ff75bcc781bc104a14f56d99 2023-07-16 21:15:43,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/Group_foo/Group_testCreateAndAssign/647cf4e5ff75bcc781bc104a14f56d99 2023-07-16 21:15:43,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 647cf4e5ff75bcc781bc104a14f56d99 2023-07-16 21:15:43,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/Group_foo/Group_testCreateAndAssign/647cf4e5ff75bcc781bc104a14f56d99/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:43,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 647cf4e5ff75bcc781bc104a14f56d99; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=9739446240, jitterRate=-0.09294338524341583}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:43,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 647cf4e5ff75bcc781bc104a14f56d99: 2023-07-16 21:15:43,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99., pid=67, masterSystemTime=1689542143029 2023-07-16 21:15:43,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99. 2023-07-16 21:15:43,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99. 2023-07-16 21:15:43,048 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=66 updating hbase:meta row=647cf4e5ff75bcc781bc104a14f56d99, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:43,048 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99.","families":{"info":[{"qualifier":"regioninfo","vlen":61,"tag":[],"timestamp":"1689542143048"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542143048"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542143048"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542143048"}]},"ts":"1689542143048"} 2023-07-16 21:15:43,052 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-07-16 21:15:43,052 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; OpenRegionProcedure 647cf4e5ff75bcc781bc104a14f56d99, server=jenkins-hbase4.apache.org,33511,1689542134438 in 173 msec 2023-07-16 21:15:43,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-07-16 21:15:43,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; TransitRegionStateProcedure table=Group_foo:Group_testCreateAndAssign, region=647cf4e5ff75bcc781bc104a14f56d99, ASSIGN in 332 msec 2023-07-16 21:15:43,055 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=65, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-16 21:15:43,055 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_foo:Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542143055"}]},"ts":"1689542143055"} 2023-07-16 21:15:43,056 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=Group_foo:Group_testCreateAndAssign, state=ENABLED in hbase:meta 2023-07-16 21:15:43,059 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=65, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign execute state=CREATE_TABLE_POST_OPERATION 2023-07-16 21:15:43,060 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign in 438 msec 2023-07-16 21:15:43,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=65 2023-07-16 21:15:43,230 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: Group_foo:Group_testCreateAndAssign, procId: 65 completed 2023-07-16 21:15:43,231 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:43,237 INFO [Listener at localhost/43605] client.HBaseAdmin$15(890): Started disable of Group_foo:Group_testCreateAndAssign 2023-07-16 21:15:43,238 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable Group_foo:Group_testCreateAndAssign 2023-07-16 21:15:43,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-16 21:15:43,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-07-16 21:15:43,243 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_foo:Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542143243"}]},"ts":"1689542143243"} 2023-07-16 21:15:43,245 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=Group_foo:Group_testCreateAndAssign, state=DISABLING in hbase:meta 2023-07-16 21:15:43,246 INFO [PEWorker-2] procedure.DisableTableProcedure(293): Set Group_foo:Group_testCreateAndAssign to state=DISABLING 2023-07-16 21:15:43,247 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=68, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_foo:Group_testCreateAndAssign, region=647cf4e5ff75bcc781bc104a14f56d99, UNASSIGN}] 2023-07-16 21:15:43,249 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=69, ppid=68, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_foo:Group_testCreateAndAssign, region=647cf4e5ff75bcc781bc104a14f56d99, UNASSIGN 2023-07-16 21:15:43,250 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=69 updating hbase:meta row=647cf4e5ff75bcc781bc104a14f56d99, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:43,251 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99.","families":{"info":[{"qualifier":"regioninfo","vlen":61,"tag":[],"timestamp":"1689542143250"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542143250"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542143250"}]},"ts":"1689542143250"} 2023-07-16 21:15:43,252 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE; CloseRegionProcedure 647cf4e5ff75bcc781bc104a14f56d99, server=jenkins-hbase4.apache.org,33511,1689542134438}] 2023-07-16 21:15:43,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-07-16 21:15:43,407 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 647cf4e5ff75bcc781bc104a14f56d99 2023-07-16 21:15:43,408 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 647cf4e5ff75bcc781bc104a14f56d99, disabling compactions & flushes 2023-07-16 21:15:43,408 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99. 2023-07-16 21:15:43,408 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99. 2023-07-16 21:15:43,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99. after waiting 0 ms 2023-07-16 21:15:43,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99. 2023-07-16 21:15:43,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/Group_foo/Group_testCreateAndAssign/647cf4e5ff75bcc781bc104a14f56d99/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:15:43,414 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99. 2023-07-16 21:15:43,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 647cf4e5ff75bcc781bc104a14f56d99: 2023-07-16 21:15:43,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 647cf4e5ff75bcc781bc104a14f56d99 2023-07-16 21:15:43,417 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=69 updating hbase:meta row=647cf4e5ff75bcc781bc104a14f56d99, regionState=CLOSED 2023-07-16 21:15:43,417 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99.","families":{"info":[{"qualifier":"regioninfo","vlen":61,"tag":[],"timestamp":"1689542143416"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542143416"}]},"ts":"1689542143416"} 2023-07-16 21:15:43,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-07-16 21:15:43,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; CloseRegionProcedure 647cf4e5ff75bcc781bc104a14f56d99, server=jenkins-hbase4.apache.org,33511,1689542134438 in 166 msec 2023-07-16 21:15:43,423 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=68 2023-07-16 21:15:43,423 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=68, state=SUCCESS; TransitRegionStateProcedure table=Group_foo:Group_testCreateAndAssign, region=647cf4e5ff75bcc781bc104a14f56d99, UNASSIGN in 174 msec 2023-07-16 21:15:43,425 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_foo:Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542143425"}]},"ts":"1689542143425"} 2023-07-16 21:15:43,427 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=Group_foo:Group_testCreateAndAssign, state=DISABLED in hbase:meta 2023-07-16 21:15:43,430 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set Group_foo:Group_testCreateAndAssign to state=DISABLED 2023-07-16 21:15:43,432 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DisableTableProcedure table=Group_foo:Group_testCreateAndAssign in 193 msec 2023-07-16 21:15:43,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-07-16 21:15:43,545 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: Group_foo:Group_testCreateAndAssign, procId: 68 completed 2023-07-16 21:15:43,546 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete Group_foo:Group_testCreateAndAssign 2023-07-16 21:15:43,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-16 21:15:43,549 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=71, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-16 21:15:43,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint(577): Removing deleted table 'Group_foo:Group_testCreateAndAssign' from rsgroup 'appInfo' 2023-07-16 21:15:43,550 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=71, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-16 21:15:43,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:43,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:43,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-16 21:15:43,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:15:43,555 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_foo/Group_testCreateAndAssign/647cf4e5ff75bcc781bc104a14f56d99 2023-07-16 21:15:43,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-07-16 21:15:43,558 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_foo/Group_testCreateAndAssign/647cf4e5ff75bcc781bc104a14f56d99/f, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_foo/Group_testCreateAndAssign/647cf4e5ff75bcc781bc104a14f56d99/recovered.edits] 2023-07-16 21:15:43,565 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_foo/Group_testCreateAndAssign/647cf4e5ff75bcc781bc104a14f56d99/recovered.edits/4.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/Group_foo/Group_testCreateAndAssign/647cf4e5ff75bcc781bc104a14f56d99/recovered.edits/4.seqid 2023-07-16 21:15:43,565 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_foo/Group_testCreateAndAssign/647cf4e5ff75bcc781bc104a14f56d99 2023-07-16 21:15:43,565 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived Group_foo:Group_testCreateAndAssign regions 2023-07-16 21:15:43,579 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=71, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-16 21:15:43,582 WARN [PEWorker-4] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of Group_foo:Group_testCreateAndAssign from hbase:meta 2023-07-16 21:15:43,585 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(421): Removing 'Group_foo:Group_testCreateAndAssign' descriptor. 2023-07-16 21:15:43,587 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=71, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-16 21:15:43,587 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(411): Removing 'Group_foo:Group_testCreateAndAssign' from region states. 2023-07-16 21:15:43,587 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542143587"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:43,589 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-16 21:15:43,590 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 647cf4e5ff75bcc781bc104a14f56d99, NAME => 'Group_foo:Group_testCreateAndAssign,,1689542142620.647cf4e5ff75bcc781bc104a14f56d99.', STARTKEY => '', ENDKEY => ''}] 2023-07-16 21:15:43,590 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(415): Marking 'Group_foo:Group_testCreateAndAssign' as deleted. 2023-07-16 21:15:43,590 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_foo:Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689542143590"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:43,592 INFO [PEWorker-4] hbase.MetaTableAccessor(1658): Deleted table Group_foo:Group_testCreateAndAssign state from META 2023-07-16 21:15:43,594 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(130): Finished pid=71, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-16 21:15:43,598 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign in 48 msec 2023-07-16 21:15:43,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-07-16 21:15:43,659 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: Group_foo:Group_testCreateAndAssign, procId: 71 completed 2023-07-16 21:15:43,666 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$17(3086): Client=jenkins//172.31.14.131 delete Group_foo 2023-07-16 21:15:43,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:DELETE_NAMESPACE_PREPARE; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-16 21:15:43,675 INFO [PEWorker-3] procedure.DeleteNamespaceProcedure(73): pid=72, state=RUNNABLE:DELETE_NAMESPACE_PREPARE, locked=true; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-16 21:15:43,678 INFO [PEWorker-3] procedure.DeleteNamespaceProcedure(73): pid=72, state=RUNNABLE:DELETE_NAMESPACE_DELETE_FROM_NS_TABLE, locked=true; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-16 21:15:43,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=72 2023-07-16 21:15:43,682 INFO [PEWorker-3] procedure.DeleteNamespaceProcedure(73): pid=72, state=RUNNABLE:DELETE_NAMESPACE_REMOVE_FROM_ZK, locked=true; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-16 21:15:43,683 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/namespace/Group_foo 2023-07-16 21:15:43,683 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-16 21:15:43,685 INFO [PEWorker-3] procedure.DeleteNamespaceProcedure(73): pid=72, state=RUNNABLE:DELETE_NAMESPACE_DELETE_DIRECTORIES, locked=true; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-16 21:15:43,688 INFO [PEWorker-3] procedure.DeleteNamespaceProcedure(73): pid=72, state=RUNNABLE:DELETE_NAMESPACE_REMOVE_NAMESPACE_QUOTA, locked=true; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-16 21:15:43,689 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; DeleteNamespaceProcedure, namespace=Group_foo in 21 msec 2023-07-16 21:15:43,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=72 2023-07-16 21:15:43,783 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:43,784 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:43,785 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:43,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:43,785 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:43,786 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:43,787 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:43,790 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:43,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:43,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-16 21:15:43,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 5 2023-07-16 21:15:43,797 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:43,799 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:43,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:43,799 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:43,800 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:33511] to rsgroup default 2023-07-16 21:15:43,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:43,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-16 21:15:43,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:43,808 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(286): Moving 0 region(s) to group appInfo, current retry=0 2023-07-16 21:15:43,808 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(261): All regions from [jenkins-hbase4.apache.org,33511,1689542134438] are moved back to appInfo 2023-07-16 21:15:43,808 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(438): Move servers done: appInfo => default 2023-07-16 21:15:43,808 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:43,809 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup appInfo 2023-07-16 21:15:43,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:43,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:43,815 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:43,819 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:15:43,819 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:43,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:43,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:43,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:43,826 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:43,830 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:43,830 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:43,832 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:43,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:43,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.CallRunner(144): callId: 363 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:39404 deadline: 1689543343832, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:43,833 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:43,835 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:43,835 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:43,835 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:43,836 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:43,837 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:43,837 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:43,858 INFO [Listener at localhost/43605] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testNamespaceCreateAndAssign Thread=505 (was 503) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-870215550_17 at /127.0.0.1:55766 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=771 (was 773), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=312 (was 312), ProcessCount=172 (was 172), AvailableMemoryMB=2795 (was 2829) 2023-07-16 21:15:43,858 WARN [Listener at localhost/43605] hbase.ResourceChecker(130): Thread=505 is superior to 500 2023-07-16 21:15:43,876 INFO [Listener at localhost/43605] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testCreateAndDrop Thread=505, OpenFileDescriptor=771, MaxFileDescriptor=60000, SystemLoadAverage=312, ProcessCount=172, AvailableMemoryMB=2795 2023-07-16 21:15:43,876 WARN [Listener at localhost/43605] hbase.ResourceChecker(130): Thread=505 is superior to 500 2023-07-16 21:15:43,876 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(132): testCreateAndDrop 2023-07-16 21:15:43,881 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:43,882 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:43,883 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:43,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:43,883 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:43,884 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:43,884 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:43,887 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:43,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:43,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:43,893 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:43,897 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:15:43,898 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:43,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:43,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:43,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:43,903 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:43,912 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:43,913 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:43,915 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:43,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:43,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.CallRunner(144): callId: 391 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:39404 deadline: 1689543343915, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:43,916 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:43,918 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:43,919 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:43,919 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:43,919 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:43,920 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:43,921 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:43,924 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'Group_testCreateAndDrop', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'cf', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-16 21:15:43,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=73, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=Group_testCreateAndDrop 2023-07-16 21:15:43,928 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=73, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=Group_testCreateAndDrop execute state=CREATE_TABLE_PRE_OPERATION 2023-07-16 21:15:43,929 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "Group_testCreateAndDrop" procId is: 73 2023-07-16 21:15:43,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=73 2023-07-16 21:15:43,932 DEBUG [PEWorker-1] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:43,932 DEBUG [PEWorker-1] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:43,933 DEBUG [PEWorker-1] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:43,938 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=73, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=Group_testCreateAndDrop execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-16 21:15:43,940 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndDrop/da0c51cc67c34692e2a7a17d7e1a6049 2023-07-16 21:15:43,941 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndDrop/da0c51cc67c34692e2a7a17d7e1a6049 empty. 2023-07-16 21:15:43,941 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndDrop/da0c51cc67c34692e2a7a17d7e1a6049 2023-07-16 21:15:43,941 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived Group_testCreateAndDrop regions 2023-07-16 21:15:43,966 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndDrop/.tabledesc/.tableinfo.0000000001 2023-07-16 21:15:43,968 INFO [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(7675): creating {ENCODED => da0c51cc67c34692e2a7a17d7e1a6049, NAME => 'Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='Group_testCreateAndDrop', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'cf', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:43,985 DEBUG [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(866): Instantiated Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:43,985 DEBUG [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(1604): Closing da0c51cc67c34692e2a7a17d7e1a6049, disabling compactions & flushes 2023-07-16 21:15:43,985 INFO [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(1626): Closing region Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049. 2023-07-16 21:15:43,986 DEBUG [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049. 2023-07-16 21:15:43,986 DEBUG [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049. after waiting 0 ms 2023-07-16 21:15:43,986 DEBUG [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049. 2023-07-16 21:15:43,986 INFO [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(1838): Closed Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049. 2023-07-16 21:15:43,986 DEBUG [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(1558): Region close journal for da0c51cc67c34692e2a7a17d7e1a6049: 2023-07-16 21:15:43,988 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=73, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=Group_testCreateAndDrop execute state=CREATE_TABLE_ADD_TO_META 2023-07-16 21:15:43,990 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1689542143989"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542143989"}]},"ts":"1689542143989"} 2023-07-16 21:15:43,992 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-16 21:15:43,994 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=73, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=Group_testCreateAndDrop execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-16 21:15:43,994 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndDrop","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542143994"}]},"ts":"1689542143994"} 2023-07-16 21:15:43,996 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndDrop, state=ENABLING in hbase:meta 2023-07-16 21:15:43,999 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-16 21:15:43,999 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-16 21:15:44,000 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-16 21:15:44,000 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-16 21:15:44,000 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-16 21:15:44,000 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-16 21:15:44,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateAndDrop, region=da0c51cc67c34692e2a7a17d7e1a6049, ASSIGN}] 2023-07-16 21:15:44,004 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=74, ppid=73, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateAndDrop, region=da0c51cc67c34692e2a7a17d7e1a6049, ASSIGN 2023-07-16 21:15:44,005 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=74, ppid=73, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateAndDrop, region=da0c51cc67c34692e2a7a17d7e1a6049, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36993,1689542134044; forceNewPlan=false, retain=false 2023-07-16 21:15:44,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=73 2023-07-16 21:15:44,155 INFO [jenkins-hbase4:41499] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-16 21:15:44,157 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=74 updating hbase:meta row=da0c51cc67c34692e2a7a17d7e1a6049, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:44,157 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1689542144157"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542144157"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542144157"}]},"ts":"1689542144157"} 2023-07-16 21:15:44,159 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=74, state=RUNNABLE; OpenRegionProcedure da0c51cc67c34692e2a7a17d7e1a6049, server=jenkins-hbase4.apache.org,36993,1689542134044}] 2023-07-16 21:15:44,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=73 2023-07-16 21:15:44,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049. 2023-07-16 21:15:44,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => da0c51cc67c34692e2a7a17d7e1a6049, NAME => 'Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:15:44,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateAndDrop da0c51cc67c34692e2a7a17d7e1a6049 2023-07-16 21:15:44,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:44,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for da0c51cc67c34692e2a7a17d7e1a6049 2023-07-16 21:15:44,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for da0c51cc67c34692e2a7a17d7e1a6049 2023-07-16 21:15:44,318 INFO [StoreOpener-da0c51cc67c34692e2a7a17d7e1a6049-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf of region da0c51cc67c34692e2a7a17d7e1a6049 2023-07-16 21:15:44,320 DEBUG [StoreOpener-da0c51cc67c34692e2a7a17d7e1a6049-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateAndDrop/da0c51cc67c34692e2a7a17d7e1a6049/cf 2023-07-16 21:15:44,320 DEBUG [StoreOpener-da0c51cc67c34692e2a7a17d7e1a6049-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateAndDrop/da0c51cc67c34692e2a7a17d7e1a6049/cf 2023-07-16 21:15:44,320 INFO [StoreOpener-da0c51cc67c34692e2a7a17d7e1a6049-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region da0c51cc67c34692e2a7a17d7e1a6049 columnFamilyName cf 2023-07-16 21:15:44,321 INFO [StoreOpener-da0c51cc67c34692e2a7a17d7e1a6049-1] regionserver.HStore(310): Store=da0c51cc67c34692e2a7a17d7e1a6049/cf, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:44,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateAndDrop/da0c51cc67c34692e2a7a17d7e1a6049 2023-07-16 21:15:44,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateAndDrop/da0c51cc67c34692e2a7a17d7e1a6049 2023-07-16 21:15:44,326 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for da0c51cc67c34692e2a7a17d7e1a6049 2023-07-16 21:15:44,328 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateAndDrop/da0c51cc67c34692e2a7a17d7e1a6049/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:44,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened da0c51cc67c34692e2a7a17d7e1a6049; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=9702640160, jitterRate=-0.09637121856212616}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:44,329 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for da0c51cc67c34692e2a7a17d7e1a6049: 2023-07-16 21:15:44,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049., pid=75, masterSystemTime=1689542144311 2023-07-16 21:15:44,331 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049. 2023-07-16 21:15:44,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049. 2023-07-16 21:15:44,333 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=74 updating hbase:meta row=da0c51cc67c34692e2a7a17d7e1a6049, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:44,333 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1689542144333"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542144333"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542144333"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542144333"}]},"ts":"1689542144333"} 2023-07-16 21:15:44,344 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=74 2023-07-16 21:15:44,345 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=74, state=SUCCESS; OpenRegionProcedure da0c51cc67c34692e2a7a17d7e1a6049, server=jenkins-hbase4.apache.org,36993,1689542134044 in 180 msec 2023-07-16 21:15:44,347 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-07-16 21:15:44,347 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateAndDrop, region=da0c51cc67c34692e2a7a17d7e1a6049, ASSIGN in 345 msec 2023-07-16 21:15:44,348 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=73, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=Group_testCreateAndDrop execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-16 21:15:44,348 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndDrop","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542144348"}]},"ts":"1689542144348"} 2023-07-16 21:15:44,350 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndDrop, state=ENABLED in hbase:meta 2023-07-16 21:15:44,352 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=73, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=Group_testCreateAndDrop execute state=CREATE_TABLE_POST_OPERATION 2023-07-16 21:15:44,355 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=73, state=SUCCESS; CreateTableProcedure table=Group_testCreateAndDrop in 428 msec 2023-07-16 21:15:44,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=73 2023-07-16 21:15:44,534 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:Group_testCreateAndDrop, procId: 73 completed 2023-07-16 21:15:44,535 DEBUG [Listener at localhost/43605] hbase.HBaseTestingUtility(3430): Waiting until all regions of table Group_testCreateAndDrop get assigned. Timeout = 60000ms 2023-07-16 21:15:44,535 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:44,539 INFO [Listener at localhost/43605] hbase.HBaseTestingUtility(3484): All regions for table Group_testCreateAndDrop assigned to meta. Checking AM states. 2023-07-16 21:15:44,539 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:44,539 INFO [Listener at localhost/43605] hbase.HBaseTestingUtility(3504): All regions for table Group_testCreateAndDrop assigned. 2023-07-16 21:15:44,540 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:44,544 INFO [Listener at localhost/43605] client.HBaseAdmin$15(890): Started disable of Group_testCreateAndDrop 2023-07-16 21:15:44,544 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable Group_testCreateAndDrop 2023-07-16 21:15:44,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=76, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=Group_testCreateAndDrop 2023-07-16 21:15:44,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=76 2023-07-16 21:15:44,548 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndDrop","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542144548"}]},"ts":"1689542144548"} 2023-07-16 21:15:44,550 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndDrop, state=DISABLING in hbase:meta 2023-07-16 21:15:44,553 INFO [PEWorker-2] procedure.DisableTableProcedure(293): Set Group_testCreateAndDrop to state=DISABLING 2023-07-16 21:15:44,553 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateAndDrop, region=da0c51cc67c34692e2a7a17d7e1a6049, UNASSIGN}] 2023-07-16 21:15:44,555 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=77, ppid=76, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateAndDrop, region=da0c51cc67c34692e2a7a17d7e1a6049, UNASSIGN 2023-07-16 21:15:44,556 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=77 updating hbase:meta row=da0c51cc67c34692e2a7a17d7e1a6049, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:44,556 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1689542144555"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542144555"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542144555"}]},"ts":"1689542144555"} 2023-07-16 21:15:44,557 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=77, state=RUNNABLE; CloseRegionProcedure da0c51cc67c34692e2a7a17d7e1a6049, server=jenkins-hbase4.apache.org,36993,1689542134044}] 2023-07-16 21:15:44,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=76 2023-07-16 21:15:44,709 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close da0c51cc67c34692e2a7a17d7e1a6049 2023-07-16 21:15:44,710 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing da0c51cc67c34692e2a7a17d7e1a6049, disabling compactions & flushes 2023-07-16 21:15:44,710 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049. 2023-07-16 21:15:44,710 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049. 2023-07-16 21:15:44,710 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049. after waiting 0 ms 2023-07-16 21:15:44,711 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049. 2023-07-16 21:15:44,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCreateAndDrop/da0c51cc67c34692e2a7a17d7e1a6049/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:15:44,716 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049. 2023-07-16 21:15:44,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for da0c51cc67c34692e2a7a17d7e1a6049: 2023-07-16 21:15:44,717 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed da0c51cc67c34692e2a7a17d7e1a6049 2023-07-16 21:15:44,718 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=77 updating hbase:meta row=da0c51cc67c34692e2a7a17d7e1a6049, regionState=CLOSED 2023-07-16 21:15:44,718 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1689542144718"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542144718"}]},"ts":"1689542144718"} 2023-07-16 21:15:44,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=77 2023-07-16 21:15:44,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=77, state=SUCCESS; CloseRegionProcedure da0c51cc67c34692e2a7a17d7e1a6049, server=jenkins-hbase4.apache.org,36993,1689542134044 in 162 msec 2023-07-16 21:15:44,723 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-07-16 21:15:44,723 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateAndDrop, region=da0c51cc67c34692e2a7a17d7e1a6049, UNASSIGN in 168 msec 2023-07-16 21:15:44,723 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndDrop","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542144723"}]},"ts":"1689542144723"} 2023-07-16 21:15:44,725 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndDrop, state=DISABLED in hbase:meta 2023-07-16 21:15:44,726 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set Group_testCreateAndDrop to state=DISABLED 2023-07-16 21:15:44,728 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=76, state=SUCCESS; DisableTableProcedure table=Group_testCreateAndDrop in 183 msec 2023-07-16 21:15:44,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=76 2023-07-16 21:15:44,850 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:Group_testCreateAndDrop, procId: 76 completed 2023-07-16 21:15:44,851 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete Group_testCreateAndDrop 2023-07-16 21:15:44,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-16 21:15:44,854 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=79, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-16 21:15:44,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint(577): Removing deleted table 'Group_testCreateAndDrop' from rsgroup 'default' 2023-07-16 21:15:44,855 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=79, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-16 21:15:44,859 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndDrop/da0c51cc67c34692e2a7a17d7e1a6049 2023-07-16 21:15:44,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:44,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:44,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:44,862 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndDrop/da0c51cc67c34692e2a7a17d7e1a6049/cf, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndDrop/da0c51cc67c34692e2a7a17d7e1a6049/recovered.edits] 2023-07-16 21:15:44,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-16 21:15:44,870 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndDrop/da0c51cc67c34692e2a7a17d7e1a6049/recovered.edits/4.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/default/Group_testCreateAndDrop/da0c51cc67c34692e2a7a17d7e1a6049/recovered.edits/4.seqid 2023-07-16 21:15:44,870 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCreateAndDrop/da0c51cc67c34692e2a7a17d7e1a6049 2023-07-16 21:15:44,870 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived Group_testCreateAndDrop regions 2023-07-16 21:15:44,873 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=79, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-16 21:15:44,875 WARN [PEWorker-4] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of Group_testCreateAndDrop from hbase:meta 2023-07-16 21:15:44,877 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(421): Removing 'Group_testCreateAndDrop' descriptor. 2023-07-16 21:15:44,878 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=79, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-16 21:15:44,879 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(411): Removing 'Group_testCreateAndDrop' from region states. 2023-07-16 21:15:44,879 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542144879"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:44,880 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-16 21:15:44,880 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => da0c51cc67c34692e2a7a17d7e1a6049, NAME => 'Group_testCreateAndDrop,,1689542143923.da0c51cc67c34692e2a7a17d7e1a6049.', STARTKEY => '', ENDKEY => ''}] 2023-07-16 21:15:44,880 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(415): Marking 'Group_testCreateAndDrop' as deleted. 2023-07-16 21:15:44,880 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateAndDrop","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689542144880"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:44,882 INFO [PEWorker-4] hbase.MetaTableAccessor(1658): Deleted table Group_testCreateAndDrop state from META 2023-07-16 21:15:44,884 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(130): Finished pid=79, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-16 21:15:44,886 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DeleteTableProcedure table=Group_testCreateAndDrop in 33 msec 2023-07-16 21:15:44,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-16 21:15:44,968 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:Group_testCreateAndDrop, procId: 79 completed 2023-07-16 21:15:44,973 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:44,973 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:44,974 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:44,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:44,974 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:44,975 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:44,975 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:44,976 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:44,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:44,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:44,982 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:44,985 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:15:44,988 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:44,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:44,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:44,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:44,996 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:45,000 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:45,000 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:45,003 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:45,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:45,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.CallRunner(144): callId: 450 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:39404 deadline: 1689543345003, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:45,004 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:45,007 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:45,008 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:45,008 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:45,009 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:45,009 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:45,009 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:45,026 INFO [Listener at localhost/43605] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testCreateAndDrop Thread=506 (was 505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1114267645_17 at /127.0.0.1:45048 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x50aa0278-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=771 (was 771), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=311 (was 312), ProcessCount=172 (was 172), AvailableMemoryMB=2775 (was 2795) 2023-07-16 21:15:45,027 WARN [Listener at localhost/43605] hbase.ResourceChecker(130): Thread=506 is superior to 500 2023-07-16 21:15:45,044 INFO [Listener at localhost/43605] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testCloneSnapshot Thread=506, OpenFileDescriptor=771, MaxFileDescriptor=60000, SystemLoadAverage=311, ProcessCount=172, AvailableMemoryMB=2774 2023-07-16 21:15:45,044 WARN [Listener at localhost/43605] hbase.ResourceChecker(130): Thread=506 is superior to 500 2023-07-16 21:15:45,044 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(132): testCloneSnapshot 2023-07-16 21:15:45,049 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:45,049 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:45,050 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:45,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:45,050 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:45,050 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:45,051 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:45,051 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:45,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:45,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:45,057 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:45,060 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:15:45,060 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:45,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:45,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:45,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:45,067 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:45,069 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:45,069 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:45,071 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:45,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:45,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.CallRunner(144): callId: 478 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:39404 deadline: 1689543345071, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:45,072 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:45,074 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:45,074 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:45,075 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:45,075 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:45,076 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:45,076 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:45,077 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'Group_testCloneSnapshot', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'test', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-16 21:15:45,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=80, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=Group_testCloneSnapshot 2023-07-16 21:15:45,080 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=80, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=Group_testCloneSnapshot execute state=CREATE_TABLE_PRE_OPERATION 2023-07-16 21:15:45,080 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "Group_testCloneSnapshot" procId is: 80 2023-07-16 21:15:45,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=80 2023-07-16 21:15:45,082 DEBUG [PEWorker-3] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:45,083 DEBUG [PEWorker-3] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:45,083 DEBUG [PEWorker-3] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:45,085 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=80, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=Group_testCloneSnapshot execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-16 21:15:45,086 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot/753d8adcf1c641862281758a2861f7bd 2023-07-16 21:15:45,087 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot/753d8adcf1c641862281758a2861f7bd empty. 2023-07-16 21:15:45,087 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot/753d8adcf1c641862281758a2861f7bd 2023-07-16 21:15:45,088 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived Group_testCloneSnapshot regions 2023-07-16 21:15:45,102 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot/.tabledesc/.tableinfo.0000000001 2023-07-16 21:15:45,104 INFO [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(7675): creating {ENCODED => 753d8adcf1c641862281758a2861f7bd, NAME => 'Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='Group_testCloneSnapshot', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'test', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:45,114 DEBUG [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(866): Instantiated Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:45,114 DEBUG [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(1604): Closing 753d8adcf1c641862281758a2861f7bd, disabling compactions & flushes 2023-07-16 21:15:45,114 INFO [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(1626): Closing region Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. 2023-07-16 21:15:45,114 DEBUG [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. 2023-07-16 21:15:45,114 DEBUG [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. after waiting 0 ms 2023-07-16 21:15:45,114 DEBUG [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. 2023-07-16 21:15:45,114 INFO [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(1838): Closed Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. 2023-07-16 21:15:45,114 DEBUG [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(1558): Region close journal for 753d8adcf1c641862281758a2861f7bd: 2023-07-16 21:15:45,116 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=80, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=Group_testCloneSnapshot execute state=CREATE_TABLE_ADD_TO_META 2023-07-16 21:15:45,117 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1689542145117"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542145117"}]},"ts":"1689542145117"} 2023-07-16 21:15:45,118 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-16 21:15:45,119 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=80, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=Group_testCloneSnapshot execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-16 21:15:45,119 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542145119"}]},"ts":"1689542145119"} 2023-07-16 21:15:45,121 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot, state=ENABLING in hbase:meta 2023-07-16 21:15:45,124 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-16 21:15:45,124 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-16 21:15:45,125 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-16 21:15:45,125 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-16 21:15:45,125 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-16 21:15:45,125 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-16 21:15:45,125 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=81, ppid=80, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCloneSnapshot, region=753d8adcf1c641862281758a2861f7bd, ASSIGN}] 2023-07-16 21:15:45,126 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=80, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCloneSnapshot, region=753d8adcf1c641862281758a2861f7bd, ASSIGN 2023-07-16 21:15:45,127 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=80, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCloneSnapshot, region=753d8adcf1c641862281758a2861f7bd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33511,1689542134438; forceNewPlan=false, retain=false 2023-07-16 21:15:45,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=80 2023-07-16 21:15:45,277 INFO [jenkins-hbase4:41499] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-16 21:15:45,279 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=753d8adcf1c641862281758a2861f7bd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,279 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1689542145279"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542145279"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542145279"}]},"ts":"1689542145279"} 2023-07-16 21:15:45,281 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; OpenRegionProcedure 753d8adcf1c641862281758a2861f7bd, server=jenkins-hbase4.apache.org,33511,1689542134438}] 2023-07-16 21:15:45,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=80 2023-07-16 21:15:45,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. 2023-07-16 21:15:45,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 753d8adcf1c641862281758a2861f7bd, NAME => 'Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:15:45,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCloneSnapshot 753d8adcf1c641862281758a2861f7bd 2023-07-16 21:15:45,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:45,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 753d8adcf1c641862281758a2861f7bd 2023-07-16 21:15:45,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 753d8adcf1c641862281758a2861f7bd 2023-07-16 21:15:45,440 INFO [StoreOpener-753d8adcf1c641862281758a2861f7bd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family test of region 753d8adcf1c641862281758a2861f7bd 2023-07-16 21:15:45,441 DEBUG [StoreOpener-753d8adcf1c641862281758a2861f7bd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCloneSnapshot/753d8adcf1c641862281758a2861f7bd/test 2023-07-16 21:15:45,442 DEBUG [StoreOpener-753d8adcf1c641862281758a2861f7bd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCloneSnapshot/753d8adcf1c641862281758a2861f7bd/test 2023-07-16 21:15:45,442 INFO [StoreOpener-753d8adcf1c641862281758a2861f7bd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 753d8adcf1c641862281758a2861f7bd columnFamilyName test 2023-07-16 21:15:45,443 INFO [StoreOpener-753d8adcf1c641862281758a2861f7bd-1] regionserver.HStore(310): Store=753d8adcf1c641862281758a2861f7bd/test, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:45,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCloneSnapshot/753d8adcf1c641862281758a2861f7bd 2023-07-16 21:15:45,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCloneSnapshot/753d8adcf1c641862281758a2861f7bd 2023-07-16 21:15:45,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 753d8adcf1c641862281758a2861f7bd 2023-07-16 21:15:45,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCloneSnapshot/753d8adcf1c641862281758a2861f7bd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:45,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 753d8adcf1c641862281758a2861f7bd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=9988465920, jitterRate=-0.06975162029266357}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:45,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 753d8adcf1c641862281758a2861f7bd: 2023-07-16 21:15:45,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd., pid=82, masterSystemTime=1689542145433 2023-07-16 21:15:45,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. 2023-07-16 21:15:45,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. 2023-07-16 21:15:45,454 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=753d8adcf1c641862281758a2861f7bd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,455 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1689542145454"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542145454"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542145454"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542145454"}]},"ts":"1689542145454"} 2023-07-16 21:15:45,458 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-07-16 21:15:45,458 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; OpenRegionProcedure 753d8adcf1c641862281758a2861f7bd, server=jenkins-hbase4.apache.org,33511,1689542134438 in 175 msec 2023-07-16 21:15:45,460 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=80 2023-07-16 21:15:45,460 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=80, state=SUCCESS; TransitRegionStateProcedure table=Group_testCloneSnapshot, region=753d8adcf1c641862281758a2861f7bd, ASSIGN in 333 msec 2023-07-16 21:15:45,460 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=80, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=Group_testCloneSnapshot execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-16 21:15:45,461 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542145460"}]},"ts":"1689542145460"} 2023-07-16 21:15:45,462 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot, state=ENABLED in hbase:meta 2023-07-16 21:15:45,465 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=80, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=Group_testCloneSnapshot execute state=CREATE_TABLE_POST_OPERATION 2023-07-16 21:15:45,466 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, state=SUCCESS; CreateTableProcedure table=Group_testCloneSnapshot in 388 msec 2023-07-16 21:15:45,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=80 2023-07-16 21:15:45,685 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:Group_testCloneSnapshot, procId: 80 completed 2023-07-16 21:15:45,685 DEBUG [Listener at localhost/43605] hbase.HBaseTestingUtility(3430): Waiting until all regions of table Group_testCloneSnapshot get assigned. Timeout = 60000ms 2023-07-16 21:15:45,685 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:45,688 INFO [Listener at localhost/43605] hbase.HBaseTestingUtility(3484): All regions for table Group_testCloneSnapshot assigned to meta. Checking AM states. 2023-07-16 21:15:45,688 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:45,689 INFO [Listener at localhost/43605] hbase.HBaseTestingUtility(3504): All regions for table Group_testCloneSnapshot assigned. 2023-07-16 21:15:45,702 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1583): Client=jenkins//172.31.14.131 snapshot request for:{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 } 2023-07-16 21:15:45,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1689542145702 (current time:1689542145702). 2023-07-16 21:15:45,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-07-16 21:15:45,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] zookeeper.ReadOnlyZKClient(139): Connect 0x2a4fd01a to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:15:45,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1796d429, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:15:45,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:15:45,715 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36714, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:15:45,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2a4fd01a to 127.0.0.1:63917 2023-07-16 21:15:45,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:15:45,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-07-16 21:15:45,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 } 2023-07-16 21:15:45,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=83, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=EXCLUSIVE 2023-07-16 21:15:45,772 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=83, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=EXCLUSIVE 2023-07-16 21:15:45,773 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=83, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=EXCLUSIVE; timeout=600000, timestamp=1689542745773 2023-07-16 21:15:45,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] snapshot.SnapshotManager(653): Started snapshot: { ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 } 2023-07-16 21:15:45,773 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot Group_testCloneSnapshot_snap C_M_SNAPSHOT_TABLE on table Group_testCloneSnapshot 2023-07-16 21:15:45,775 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=83, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=EXCLUSIVE 2023-07-16 21:15:45,777 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=83, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=EXCLUSIVE in 29 msec 2023-07-16 21:15:45,792 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=SHARED 2023-07-16 21:15:45,793 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=84, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=SHARED 2023-07-16 21:15:45,794 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=84, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=SHARED; timeout=600000, timestamp=1689542745794 2023-07-16 21:15:45,796 DEBUG [Listener at localhost/43605] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 }'' to complete. (max 20000 ms per retry) 2023-07-16 21:15:45,797 DEBUG [Listener at localhost/43605] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-07-16 21:15:45,859 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure Group_testCloneSnapshot_snap 2023-07-16 21:15:45,860 INFO [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'Group_testCloneSnapshot_snap' 2023-07-16 21:15:45,860 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-16 21:15:45,860 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'Group_testCloneSnapshot_snap' starting 'acquire' 2023-07-16 21:15:45,860 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'Group_testCloneSnapshot_snap', kicking off acquire phase on members. 2023-07-16 21:15:45,861 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,861 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,862 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-16 21:15:45,862 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-16 21:15:45,862 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-16 21:15:45,862 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-16 21:15:45,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-16 21:15:45,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-16 21:15:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:45,862 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-16 21:15:45,863 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:45,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:45,863 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-16 21:15:45,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-16 21:15:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,863 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,864 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,864 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 72 2023-07-16 21:15:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,864 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 72 2023-07-16 21:15:45,864 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot Group_testCloneSnapshot_snap from table Group_testCloneSnapshot type FLUSH 2023-07-16 21:15:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 72 2023-07-16 21:15:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 72 2023-07-16 21:15:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,864 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot Group_testCloneSnapshot_snap from table Group_testCloneSnapshot type FLUSH 2023-07-16 21:15:45,864 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot Group_testCloneSnapshot_snap from table Group_testCloneSnapshot type FLUSH 2023-07-16 21:15:45,864 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot Group_testCloneSnapshot_snap from table Group_testCloneSnapshot type FLUSH 2023-07-16 21:15:45,867 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:Group_testCloneSnapshot_snap 2023-07-16 21:15:45,867 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:Group_testCloneSnapshot_snap 2023-07-16 21:15:45,867 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:Group_testCloneSnapshot_snap 2023-07-16 21:15:45,870 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:Group_testCloneSnapshot_snap 2023-07-16 21:15:45,871 DEBUG [member: 'jenkins-hbase4.apache.org,36993,1689542134044' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'Group_testCloneSnapshot_snap' with timeout 300000ms 2023-07-16 21:15:45,871 DEBUG [member: 'jenkins-hbase4.apache.org,36993,1689542134044' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-16 21:15:45,871 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'Group_testCloneSnapshot_snap' with timeout 300000ms 2023-07-16 21:15:45,871 DEBUG [member: 'jenkins-hbase4.apache.org,41213,1689542138767' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'Group_testCloneSnapshot_snap' with timeout 300000ms 2023-07-16 21:15:45,871 DEBUG [member: 'jenkins-hbase4.apache.org,41213,1689542138767' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-16 21:15:45,871 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-16 21:15:45,871 DEBUG [member: 'jenkins-hbase4.apache.org,45229,1689542134233' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'Group_testCloneSnapshot_snap' with timeout 300000ms 2023-07-16 21:15:45,872 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'Group_testCloneSnapshot_snap' starting 'acquire' stage 2023-07-16 21:15:45,871 DEBUG [member: 'jenkins-hbase4.apache.org,36993,1689542134044' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'Group_testCloneSnapshot_snap' starting 'acquire' stage 2023-07-16 21:15:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,36993,1689542134044' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'Group_testCloneSnapshot_snap' locally acquired 2023-07-16 21:15:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'Group_testCloneSnapshot_snap' locally acquired 2023-07-16 21:15:45,872 DEBUG [member: 'jenkins-hbase4.apache.org,45229,1689542134233' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-16 21:15:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33511,1689542134438' joining acquired barrier for procedure (Group_testCloneSnapshot_snap) in zk 2023-07-16 21:15:45,872 DEBUG [member: 'jenkins-hbase4.apache.org,41213,1689542138767' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'Group_testCloneSnapshot_snap' starting 'acquire' stage 2023-07-16 21:15:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,41213,1689542138767' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'Group_testCloneSnapshot_snap' locally acquired 2023-07-16 21:15:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,45229,1689542134233' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'Group_testCloneSnapshot_snap' starting 'acquire' stage 2023-07-16 21:15:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,45229,1689542134233' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'Group_testCloneSnapshot_snap' locally acquired 2023-07-16 21:15:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,45229,1689542134233' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45229,1689542134233' joining acquired barrier for procedure (Group_testCloneSnapshot_snap) in zk 2023-07-16 21:15:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,36993,1689542134044' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36993,1689542134044' joining acquired barrier for procedure (Group_testCloneSnapshot_snap) in zk 2023-07-16 21:15:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,41213,1689542138767' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41213,1689542138767' joining acquired barrier for procedure (Group_testCloneSnapshot_snap) in zk 2023-07-16 21:15:45,875 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,875 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-16 21:15:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-16 21:15:45,884 DEBUG [member: 'jenkins-hbase4.apache.org,36993,1689542134044' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,884 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-16 21:15:45,884 DEBUG [member: 'jenkins-hbase4.apache.org,45229,1689542134233' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,884 DEBUG [member: 'jenkins-hbase4.apache.org,41213,1689542138767' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,884 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'Group_testCloneSnapshot_snap' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-16 21:15:45,885 DEBUG [member: 'jenkins-hbase4.apache.org,36993,1689542134044' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-16 21:15:45,885 DEBUG [member: 'jenkins-hbase4.apache.org,36993,1689542134044' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'Group_testCloneSnapshot_snap' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-16 21:15:45,885 DEBUG [member: 'jenkins-hbase4.apache.org,45229,1689542134233' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,885 DEBUG [member: 'jenkins-hbase4.apache.org,41213,1689542138767' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,885 DEBUG [member: 'jenkins-hbase4.apache.org,41213,1689542138767' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'Group_testCloneSnapshot_snap' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-16 21:15:45,885 DEBUG [member: 'jenkins-hbase4.apache.org,45229,1689542134233' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'Group_testCloneSnapshot_snap' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-16 21:15:45,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----Group_testCloneSnapshot_snap 2023-07-16 21:15:45,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:45,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:45,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:45,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-16 21:15:45,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33511,1689542134438' joining acquired barrier for procedure 'Group_testCloneSnapshot_snap' on coordinator 2023-07-16 21:15:45,887 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'Group_testCloneSnapshot_snap' starting 'in-barrier' execution. 2023-07-16 21:15:45,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@502dbae3[Count = 0] remaining members to acquire global barrier 2023-07-16 21:15:45,887 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,888 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,888 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,888 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,888 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,889 DEBUG [member: 'jenkins-hbase4.apache.org,41213,1689542138767' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'Group_testCloneSnapshot_snap' received 'reached' from coordinator. 2023-07-16 21:15:45,888 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,889 DEBUG [member: 'jenkins-hbase4.apache.org,41213,1689542138767' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'Group_testCloneSnapshot_snap' locally completed 2023-07-16 21:15:45,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,889 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-16 21:15:45,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,889 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'Group_testCloneSnapshot_snap' received 'reached' from coordinator. 2023-07-16 21:15:45,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,889 DEBUG [member: 'jenkins-hbase4.apache.org,41213,1689542138767' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'Group_testCloneSnapshot_snap' completed for member 'jenkins-hbase4.apache.org,41213,1689542138767' in zk 2023-07-16 21:15:45,889 DEBUG [member: 'jenkins-hbase4.apache.org,45229,1689542134233' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'Group_testCloneSnapshot_snap' received 'reached' from coordinator. 2023-07-16 21:15:45,889 DEBUG [member: 'jenkins-hbase4.apache.org,45229,1689542134233' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'Group_testCloneSnapshot_snap' locally completed 2023-07-16 21:15:45,889 DEBUG [member: 'jenkins-hbase4.apache.org,45229,1689542134233' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'Group_testCloneSnapshot_snap' completed for member 'jenkins-hbase4.apache.org,45229,1689542134233' in zk 2023-07-16 21:15:45,889 DEBUG [member: 'jenkins-hbase4.apache.org,36993,1689542134044' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'Group_testCloneSnapshot_snap' received 'reached' from coordinator. 2023-07-16 21:15:45,889 DEBUG [member: 'jenkins-hbase4.apache.org,36993,1689542134044' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'Group_testCloneSnapshot_snap' locally completed 2023-07-16 21:15:45,889 DEBUG [member: 'jenkins-hbase4.apache.org,36993,1689542134044' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'Group_testCloneSnapshot_snap' completed for member 'jenkins-hbase4.apache.org,36993,1689542134044' in zk 2023-07-16 21:15:45,890 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 1 regions 2023-07-16 21:15:45,890 DEBUG [rs(jenkins-hbase4.apache.org,33511,1689542134438)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. 2023-07-16 21:15:45,890 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-07-16 21:15:45,890 DEBUG [member: 'jenkins-hbase4.apache.org,41213,1689542138767' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'Group_testCloneSnapshot_snap' has notified controller of completion 2023-07-16 21:15:45,891 DEBUG [member: 'jenkins-hbase4.apache.org,41213,1689542138767' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-16 21:15:45,891 DEBUG [member: 'jenkins-hbase4.apache.org,41213,1689542138767' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'Group_testCloneSnapshot_snap' completed. 2023-07-16 21:15:45,891 DEBUG [rs(jenkins-hbase4.apache.org,33511,1689542134438)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. started... 2023-07-16 21:15:45,893 DEBUG [member: 'jenkins-hbase4.apache.org,45229,1689542134233' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'Group_testCloneSnapshot_snap' has notified controller of completion 2023-07-16 21:15:45,893 DEBUG [member: 'jenkins-hbase4.apache.org,45229,1689542134233' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-16 21:15:45,893 DEBUG [member: 'jenkins-hbase4.apache.org,36993,1689542134044' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'Group_testCloneSnapshot_snap' has notified controller of completion 2023-07-16 21:15:45,893 DEBUG [member: 'jenkins-hbase4.apache.org,36993,1689542134044' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-16 21:15:45,893 DEBUG [member: 'jenkins-hbase4.apache.org,45229,1689542134233' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'Group_testCloneSnapshot_snap' completed. 2023-07-16 21:15:45,894 DEBUG [member: 'jenkins-hbase4.apache.org,36993,1689542134044' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'Group_testCloneSnapshot_snap' completed. 2023-07-16 21:15:45,896 DEBUG [rs(jenkins-hbase4.apache.org,33511,1689542134438)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for 753d8adcf1c641862281758a2861f7bd: 2023-07-16 21:15:45,897 DEBUG [Listener at localhost/43605] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-16 21:15:45,897 DEBUG [rs(jenkins-hbase4.apache.org,33511,1689542134438)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd.' region-info for snapshot=Group_testCloneSnapshot_snap 2023-07-16 21:15:45,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 } is done 2023-07-16 21:15:45,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] snapshot.SnapshotManager(404): Snapshoting '{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 }' is still in progress! 2023-07-16 21:15:45,901 DEBUG [Listener at localhost/43605] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-07-16 21:15:45,903 DEBUG [rs(jenkins-hbase4.apache.org,33511,1689542134438)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-07-16 21:15:45,907 DEBUG [rs(jenkins-hbase4.apache.org,33511,1689542134438)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [] hfiles 2023-07-16 21:15:45,923 DEBUG [rs(jenkins-hbase4.apache.org,33511,1689542134438)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. completed. 2023-07-16 21:15:45,923 DEBUG [rs(jenkins-hbase4.apache.org,33511,1689542134438)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. 2023-07-16 21:15:45,924 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/1 local region snapshots. 2023-07-16 21:15:45,924 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 1 local region snapshots. 2023-07-16 21:15:45,924 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,924 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'Group_testCloneSnapshot_snap' locally completed 2023-07-16 21:15:45,924 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'Group_testCloneSnapshot_snap' completed for member 'jenkins-hbase4.apache.org,33511,1689542134438' in zk 2023-07-16 21:15:45,925 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,925 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'Group_testCloneSnapshot_snap' has notified controller of completion 2023-07-16 21:15:45,926 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-16 21:15:45,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,926 DEBUG [member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'Group_testCloneSnapshot_snap' completed. 2023-07-16 21:15:45,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-16 21:15:45,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-16 21:15:45,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-16 21:15:45,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-16 21:15:45,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----Group_testCloneSnapshot_snap 2023-07-16 21:15:45,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:45,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:45,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:45,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-16 21:15:45,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----Group_testCloneSnapshot_snap 2023-07-16 21:15:45,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:45,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:45,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:45,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'Group_testCloneSnapshot_snap' member 'jenkins-hbase4.apache.org,33511,1689542134438': 2023-07-16 21:15:45,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33511,1689542134438' released barrier for procedure'Group_testCloneSnapshot_snap', counting down latch. Waiting for 0 more 2023-07-16 21:15:45,931 INFO [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'Group_testCloneSnapshot_snap' execution completed 2023-07-16 21:15:45,931 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-16 21:15:45,931 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-16 21:15:45,931 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:Group_testCloneSnapshot_snap 2023-07-16 21:15:45,931 INFO [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure Group_testCloneSnapshot_snapincluding nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-07-16 21:15:45,944 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,944 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,944 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-16 21:15:45,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-16 21:15:45,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,944 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,944 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-16 21:15:45,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,944 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-16 21:15:45,944 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-16 21:15:45,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,945 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:45,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-16 21:15:45,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-16 21:15:45,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:45,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-16 21:15:45,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-16 21:15:45,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:45,946 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:45,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----Group_testCloneSnapshot_snap 2023-07-16 21:15:45,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,946 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:45,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-16 21:15:45,947 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:45,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----Group_testCloneSnapshot_snap 2023-07-16 21:15:45,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:45,948 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,948 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-16 21:15:45,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-16 21:15:45,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:45,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:45,950 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:45,950 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,951 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:45,951 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:45,956 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:45,956 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-16 21:15:45,956 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-16 21:15:45,956 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-16 21:15:45,956 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-16 21:15:45,956 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-16 21:15:45,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:45,956 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-16 21:15:45,956 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-16 21:15:45,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-16 21:15:45,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-16 21:15:45,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:45,957 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-16 21:15:45,957 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:45,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:45,957 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-16 21:15:45,957 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,957 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:45,957 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,957 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:45,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-16 21:15:45,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-16 21:15:45,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:45,958 DEBUG [(jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-16 21:15:45,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-16 21:15:45,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:45,957 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,958 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for Group_testCloneSnapshot_snap 2023-07-16 21:15:45,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:45,960 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:45,960 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:45,960 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:45,960 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for Group_testCloneSnapshot_snap 2023-07-16 21:15:45,960 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,960 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:45,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-16 21:15:45,962 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap because node does not exist (not an error) 2023-07-16 21:15:45,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-16 21:15:45,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:45,963 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.hbase-snapshot/.tmp/Group_testCloneSnapshot_snap 2023-07-16 21:15:45,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-16 21:15:45,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:45,989 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap because node does not exist (not an error) 2023-07-16 21:15:46,088 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.hbase-snapshot/.tmp/Group_testCloneSnapshot_snap to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.hbase-snapshot/Group_testCloneSnapshot_snap 2023-07-16 21:15:46,101 DEBUG [Listener at localhost/43605] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-16 21:15:46,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 } is done 2023-07-16 21:15:46,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] snapshot.SnapshotManager(404): Snapshoting '{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 }' is still in progress! 2023-07-16 21:15:46,103 DEBUG [Listener at localhost/43605] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-07-16 21:15:46,140 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot Group_testCloneSnapshot_snap of table Group_testCloneSnapshot completed 2023-07-16 21:15:46,141 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.hbase-snapshot/.tmp/Group_testCloneSnapshot_snap 2023-07-16 21:15:46,141 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.hbase-snapshot/.tmp/Group_testCloneSnapshot_snap 2023-07-16 21:15:46,141 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot Group_testCloneSnapshot_snap C_M_SNAPSHOT_TABLE on table Group_testCloneSnapshot at 1689542145773Consolidate snapshot: Group_testCloneSnapshot_snap at 1689542145960 (+187 ms)Loading Region manifests for Group_testCloneSnapshot_snap at 1689542145960Writing data manifest for Group_testCloneSnapshot_snap at 1689542145987 (+27 ms)Verifying snapshot: Group_testCloneSnapshot_snap at 1689542146076 (+89 ms)Snapshot Group_testCloneSnapshot_snap of table Group_testCloneSnapshot completed at 1689542146140 (+64 ms) 2023-07-16 21:15:46,143 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=84, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=SHARED 2023-07-16 21:15:46,145 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=SHARED in 352 msec 2023-07-16 21:15:46,403 DEBUG [Listener at localhost/43605] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-16 21:15:46,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 } is done 2023-07-16 21:15:46,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] snapshot.SnapshotManager(401): Snapshot '{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 }' has completed, notifying client. 2023-07-16 21:15:46,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint(486): Pre-moving table Group_testCloneSnapshot_clone to RSGroup default 2023-07-16 21:15:46,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:46,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:46,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:46,421 ERROR [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(742): TableDescriptor of table {} not found. Skipping the region movement of this table. 2023-07-16 21:15:46,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CLONE_SNAPSHOT_PRE_OPERATION; CloneSnapshotProcedure (table=Group_testCloneSnapshot_clone snapshot=name: "Group_testCloneSnapshot_snap" table: "Group_testCloneSnapshot" creation_time: 1689542145702 type: FLUSH version: 2 ttl: 0 ) 2023-07-16 21:15:46,434 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] snapshot.SnapshotManager(750): Clone snapshot=Group_testCloneSnapshot_snap as table=Group_testCloneSnapshot_clone 2023-07-16 21:15:46,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=85 2023-07-16 21:15:46,454 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot_clone/.tabledesc/.tableinfo.0000000001 2023-07-16 21:15:46,461 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "Group_testCloneSnapshot_snap" table: "Group_testCloneSnapshot" creation_time: 1689542145702 type: FLUSH version: 2 ttl: 0 2023-07-16 21:15:46,462 DEBUG [PEWorker-1] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot_clone 2023-07-16 21:15:46,463 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(239): region to add: 753d8adcf1c641862281758a2861f7bd 2023-07-16 21:15:46,463 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(585): clone region=753d8adcf1c641862281758a2861f7bd as b027f6d2ab84e31a2f4bb10a377a7d43 in snapshot Group_testCloneSnapshot_snap 2023-07-16 21:15:46,464 INFO [RestoreSnapshot-pool-0] regionserver.HRegion(7675): creating {ENCODED => b027f6d2ab84e31a2f4bb10a377a7d43, NAME => 'Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='Group_testCloneSnapshot_clone', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'test', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:46,479 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(866): Instantiated Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:46,479 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(1604): Closing b027f6d2ab84e31a2f4bb10a377a7d43, disabling compactions & flushes 2023-07-16 21:15:46,479 INFO [RestoreSnapshot-pool-0] regionserver.HRegion(1626): Closing region Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43. 2023-07-16 21:15:46,479 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43. 2023-07-16 21:15:46,479 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43. after waiting 0 ms 2023-07-16 21:15:46,479 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43. 2023-07-16 21:15:46,479 INFO [RestoreSnapshot-pool-0] regionserver.HRegion(1838): Closed Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43. 2023-07-16 21:15:46,479 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(1558): Region close journal for b027f6d2ab84e31a2f4bb10a377a7d43: 2023-07-16 21:15:46,479 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "Group_testCloneSnapshot_snap" table: "Group_testCloneSnapshot" creation_time: 1689542145702 type: FLUSH version: 2 ttl: 0 2023-07-16 21:15:46,480 INFO [PEWorker-1] procedure.CloneSnapshotProcedure$1(421): Clone snapshot=Group_testCloneSnapshot_snap on table=Group_testCloneSnapshot_clone completed! 2023-07-16 21:15:46,485 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1689542146485"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542146485"}]},"ts":"1689542146485"} 2023-07-16 21:15:46,487 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-16 21:15:46,488 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot_clone","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542146488"}]},"ts":"1689542146488"} 2023-07-16 21:15:46,489 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot_clone, state=ENABLING in hbase:meta 2023-07-16 21:15:46,494 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-16 21:15:46,494 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-16 21:15:46,494 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-16 21:15:46,494 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-16 21:15:46,494 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-16 21:15:46,494 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-16 21:15:46,494 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCloneSnapshot_clone, region=b027f6d2ab84e31a2f4bb10a377a7d43, ASSIGN}] 2023-07-16 21:15:46,496 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCloneSnapshot_clone, region=b027f6d2ab84e31a2f4bb10a377a7d43, ASSIGN 2023-07-16 21:15:46,497 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCloneSnapshot_clone, region=b027f6d2ab84e31a2f4bb10a377a7d43, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33511,1689542134438; forceNewPlan=false, retain=false 2023-07-16 21:15:46,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=85 2023-07-16 21:15:46,647 INFO [jenkins-hbase4:41499] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-16 21:15:46,648 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=b027f6d2ab84e31a2f4bb10a377a7d43, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:46,649 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1689542146648"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542146648"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542146648"}]},"ts":"1689542146648"} 2023-07-16 21:15:46,650 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure b027f6d2ab84e31a2f4bb10a377a7d43, server=jenkins-hbase4.apache.org,33511,1689542134438}] 2023-07-16 21:15:46,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=85 2023-07-16 21:15:46,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43. 2023-07-16 21:15:46,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b027f6d2ab84e31a2f4bb10a377a7d43, NAME => 'Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:15:46,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCloneSnapshot_clone b027f6d2ab84e31a2f4bb10a377a7d43 2023-07-16 21:15:46,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:46,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b027f6d2ab84e31a2f4bb10a377a7d43 2023-07-16 21:15:46,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b027f6d2ab84e31a2f4bb10a377a7d43 2023-07-16 21:15:46,808 INFO [StoreOpener-b027f6d2ab84e31a2f4bb10a377a7d43-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family test of region b027f6d2ab84e31a2f4bb10a377a7d43 2023-07-16 21:15:46,809 DEBUG [StoreOpener-b027f6d2ab84e31a2f4bb10a377a7d43-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCloneSnapshot_clone/b027f6d2ab84e31a2f4bb10a377a7d43/test 2023-07-16 21:15:46,809 DEBUG [StoreOpener-b027f6d2ab84e31a2f4bb10a377a7d43-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCloneSnapshot_clone/b027f6d2ab84e31a2f4bb10a377a7d43/test 2023-07-16 21:15:46,810 INFO [StoreOpener-b027f6d2ab84e31a2f4bb10a377a7d43-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b027f6d2ab84e31a2f4bb10a377a7d43 columnFamilyName test 2023-07-16 21:15:46,810 INFO [StoreOpener-b027f6d2ab84e31a2f4bb10a377a7d43-1] regionserver.HStore(310): Store=b027f6d2ab84e31a2f4bb10a377a7d43/test, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:46,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCloneSnapshot_clone/b027f6d2ab84e31a2f4bb10a377a7d43 2023-07-16 21:15:46,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCloneSnapshot_clone/b027f6d2ab84e31a2f4bb10a377a7d43 2023-07-16 21:15:46,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b027f6d2ab84e31a2f4bb10a377a7d43 2023-07-16 21:15:46,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCloneSnapshot_clone/b027f6d2ab84e31a2f4bb10a377a7d43/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:46,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b027f6d2ab84e31a2f4bb10a377a7d43; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11439096160, jitterRate=0.06534884870052338}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:46,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b027f6d2ab84e31a2f4bb10a377a7d43: 2023-07-16 21:15:46,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43., pid=87, masterSystemTime=1689542146802 2023-07-16 21:15:46,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43. 2023-07-16 21:15:46,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43. 2023-07-16 21:15:46,819 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=b027f6d2ab84e31a2f4bb10a377a7d43, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:46,819 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1689542146819"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542146819"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542146819"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542146819"}]},"ts":"1689542146819"} 2023-07-16 21:15:46,822 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-07-16 21:15:46,823 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure b027f6d2ab84e31a2f4bb10a377a7d43, server=jenkins-hbase4.apache.org,33511,1689542134438 in 171 msec 2023-07-16 21:15:46,824 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-07-16 21:15:46,824 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=Group_testCloneSnapshot_clone, region=b027f6d2ab84e31a2f4bb10a377a7d43, ASSIGN in 329 msec 2023-07-16 21:15:46,825 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot_clone","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542146825"}]},"ts":"1689542146825"} 2023-07-16 21:15:46,827 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot_clone, state=ENABLED in hbase:meta 2023-07-16 21:15:46,831 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CloneSnapshotProcedure (table=Group_testCloneSnapshot_clone snapshot=name: "Group_testCloneSnapshot_snap" table: "Group_testCloneSnapshot" creation_time: 1689542145702 type: FLUSH version: 2 ttl: 0 ) in 403 msec 2023-07-16 21:15:47,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=85 2023-07-16 21:15:47,166 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:Group_testCloneSnapshot_clone, procId: 85 completed 2023-07-16 21:15:47,169 INFO [Listener at localhost/43605] client.HBaseAdmin$15(890): Started disable of Group_testCloneSnapshot 2023-07-16 21:15:47,169 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable Group_testCloneSnapshot 2023-07-16 21:15:47,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=Group_testCloneSnapshot 2023-07-16 21:15:47,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-16 21:15:47,178 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542147178"}]},"ts":"1689542147178"} 2023-07-16 21:15:47,180 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot, state=DISABLING in hbase:meta 2023-07-16 21:15:47,182 INFO [PEWorker-2] procedure.DisableTableProcedure(293): Set Group_testCloneSnapshot to state=DISABLING 2023-07-16 21:15:47,182 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCloneSnapshot, region=753d8adcf1c641862281758a2861f7bd, UNASSIGN}] 2023-07-16 21:15:47,184 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCloneSnapshot, region=753d8adcf1c641862281758a2861f7bd, UNASSIGN 2023-07-16 21:15:47,185 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=753d8adcf1c641862281758a2861f7bd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:47,185 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1689542147185"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542147185"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542147185"}]},"ts":"1689542147185"} 2023-07-16 21:15:47,187 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE; CloseRegionProcedure 753d8adcf1c641862281758a2861f7bd, server=jenkins-hbase4.apache.org,33511,1689542134438}] 2023-07-16 21:15:47,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-16 21:15:47,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 753d8adcf1c641862281758a2861f7bd 2023-07-16 21:15:47,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 753d8adcf1c641862281758a2861f7bd, disabling compactions & flushes 2023-07-16 21:15:47,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. 2023-07-16 21:15:47,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. 2023-07-16 21:15:47,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. after waiting 0 ms 2023-07-16 21:15:47,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. 2023-07-16 21:15:47,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCloneSnapshot/753d8adcf1c641862281758a2861f7bd/recovered.edits/5.seqid, newMaxSeqId=5, maxSeqId=1 2023-07-16 21:15:47,344 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd. 2023-07-16 21:15:47,344 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 753d8adcf1c641862281758a2861f7bd: 2023-07-16 21:15:47,346 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 753d8adcf1c641862281758a2861f7bd 2023-07-16 21:15:47,346 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=753d8adcf1c641862281758a2861f7bd, regionState=CLOSED 2023-07-16 21:15:47,346 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1689542147346"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542147346"}]},"ts":"1689542147346"} 2023-07-16 21:15:47,349 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-07-16 21:15:47,349 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; CloseRegionProcedure 753d8adcf1c641862281758a2861f7bd, server=jenkins-hbase4.apache.org,33511,1689542134438 in 161 msec 2023-07-16 21:15:47,351 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-07-16 21:15:47,351 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=Group_testCloneSnapshot, region=753d8adcf1c641862281758a2861f7bd, UNASSIGN in 167 msec 2023-07-16 21:15:47,351 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542147351"}]},"ts":"1689542147351"} 2023-07-16 21:15:47,352 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot, state=DISABLED in hbase:meta 2023-07-16 21:15:47,357 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set Group_testCloneSnapshot to state=DISABLED 2023-07-16 21:15:47,359 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=Group_testCloneSnapshot in 189 msec 2023-07-16 21:15:47,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-16 21:15:47,480 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:Group_testCloneSnapshot, procId: 88 completed 2023-07-16 21:15:47,481 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete Group_testCloneSnapshot 2023-07-16 21:15:47,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=91, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-16 21:15:47,484 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=91, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-16 21:15:47,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint(577): Removing deleted table 'Group_testCloneSnapshot' from rsgroup 'default' 2023-07-16 21:15:47,484 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=91, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-16 21:15:47,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:47,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:47,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:47,489 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot/753d8adcf1c641862281758a2861f7bd 2023-07-16 21:15:47,491 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot/753d8adcf1c641862281758a2861f7bd/recovered.edits, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot/753d8adcf1c641862281758a2861f7bd/test] 2023-07-16 21:15:47,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=91 2023-07-16 21:15:47,496 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot/753d8adcf1c641862281758a2861f7bd/recovered.edits/5.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/default/Group_testCloneSnapshot/753d8adcf1c641862281758a2861f7bd/recovered.edits/5.seqid 2023-07-16 21:15:47,498 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot/753d8adcf1c641862281758a2861f7bd 2023-07-16 21:15:47,498 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived Group_testCloneSnapshot regions 2023-07-16 21:15:47,501 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=91, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-16 21:15:47,502 WARN [PEWorker-4] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of Group_testCloneSnapshot from hbase:meta 2023-07-16 21:15:47,504 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(421): Removing 'Group_testCloneSnapshot' descriptor. 2023-07-16 21:15:47,506 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=91, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-16 21:15:47,506 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(411): Removing 'Group_testCloneSnapshot' from region states. 2023-07-16 21:15:47,506 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542147506"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:47,507 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-16 21:15:47,508 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 753d8adcf1c641862281758a2861f7bd, NAME => 'Group_testCloneSnapshot,,1689542145077.753d8adcf1c641862281758a2861f7bd.', STARTKEY => '', ENDKEY => ''}] 2023-07-16 21:15:47,508 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(415): Marking 'Group_testCloneSnapshot' as deleted. 2023-07-16 21:15:47,508 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCloneSnapshot","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689542147508"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:47,509 INFO [PEWorker-4] hbase.MetaTableAccessor(1658): Deleted table Group_testCloneSnapshot state from META 2023-07-16 21:15:47,511 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(130): Finished pid=91, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-16 21:15:47,512 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=91, state=SUCCESS; DeleteTableProcedure table=Group_testCloneSnapshot in 30 msec 2023-07-16 21:15:47,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=91 2023-07-16 21:15:47,594 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:Group_testCloneSnapshot, procId: 91 completed 2023-07-16 21:15:47,595 INFO [Listener at localhost/43605] client.HBaseAdmin$15(890): Started disable of Group_testCloneSnapshot_clone 2023-07-16 21:15:47,596 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable Group_testCloneSnapshot_clone 2023-07-16 21:15:47,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=92, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=Group_testCloneSnapshot_clone 2023-07-16 21:15:47,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=92 2023-07-16 21:15:47,600 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot_clone","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542147600"}]},"ts":"1689542147600"} 2023-07-16 21:15:47,601 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot_clone, state=DISABLING in hbase:meta 2023-07-16 21:15:47,602 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set Group_testCloneSnapshot_clone to state=DISABLING 2023-07-16 21:15:47,603 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=92, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCloneSnapshot_clone, region=b027f6d2ab84e31a2f4bb10a377a7d43, UNASSIGN}] 2023-07-16 21:15:47,604 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=92, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCloneSnapshot_clone, region=b027f6d2ab84e31a2f4bb10a377a7d43, UNASSIGN 2023-07-16 21:15:47,605 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=b027f6d2ab84e31a2f4bb10a377a7d43, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:47,605 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1689542147604"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542147604"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542147604"}]},"ts":"1689542147604"} 2023-07-16 21:15:47,606 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; CloseRegionProcedure b027f6d2ab84e31a2f4bb10a377a7d43, server=jenkins-hbase4.apache.org,33511,1689542134438}] 2023-07-16 21:15:47,666 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-16 21:15:47,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=92 2023-07-16 21:15:47,759 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close b027f6d2ab84e31a2f4bb10a377a7d43 2023-07-16 21:15:47,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b027f6d2ab84e31a2f4bb10a377a7d43, disabling compactions & flushes 2023-07-16 21:15:47,761 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43. 2023-07-16 21:15:47,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43. 2023-07-16 21:15:47,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43. after waiting 0 ms 2023-07-16 21:15:47,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43. 2023-07-16 21:15:47,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/default/Group_testCloneSnapshot_clone/b027f6d2ab84e31a2f4bb10a377a7d43/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:15:47,768 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43. 2023-07-16 21:15:47,768 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b027f6d2ab84e31a2f4bb10a377a7d43: 2023-07-16 21:15:47,770 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed b027f6d2ab84e31a2f4bb10a377a7d43 2023-07-16 21:15:47,770 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=b027f6d2ab84e31a2f4bb10a377a7d43, regionState=CLOSED 2023-07-16 21:15:47,770 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1689542147770"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542147770"}]},"ts":"1689542147770"} 2023-07-16 21:15:47,773 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-07-16 21:15:47,773 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; CloseRegionProcedure b027f6d2ab84e31a2f4bb10a377a7d43, server=jenkins-hbase4.apache.org,33511,1689542134438 in 166 msec 2023-07-16 21:15:47,775 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=92 2023-07-16 21:15:47,775 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=92, state=SUCCESS; TransitRegionStateProcedure table=Group_testCloneSnapshot_clone, region=b027f6d2ab84e31a2f4bb10a377a7d43, UNASSIGN in 170 msec 2023-07-16 21:15:47,776 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot_clone","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542147776"}]},"ts":"1689542147776"} 2023-07-16 21:15:47,777 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot_clone, state=DISABLED in hbase:meta 2023-07-16 21:15:47,779 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set Group_testCloneSnapshot_clone to state=DISABLED 2023-07-16 21:15:47,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, state=SUCCESS; DisableTableProcedure table=Group_testCloneSnapshot_clone in 184 msec 2023-07-16 21:15:47,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=92 2023-07-16 21:15:47,903 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:Group_testCloneSnapshot_clone, procId: 92 completed 2023-07-16 21:15:47,903 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete Group_testCloneSnapshot_clone 2023-07-16 21:15:47,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-16 21:15:47,906 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=95, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-16 21:15:47,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint(577): Removing deleted table 'Group_testCloneSnapshot_clone' from rsgroup 'default' 2023-07-16 21:15:47,907 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=95, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-16 21:15:47,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:47,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:47,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:47,911 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot_clone/b027f6d2ab84e31a2f4bb10a377a7d43 2023-07-16 21:15:47,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=95 2023-07-16 21:15:47,912 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot_clone/b027f6d2ab84e31a2f4bb10a377a7d43/recovered.edits, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot_clone/b027f6d2ab84e31a2f4bb10a377a7d43/test] 2023-07-16 21:15:47,917 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot_clone/b027f6d2ab84e31a2f4bb10a377a7d43/recovered.edits/4.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/default/Group_testCloneSnapshot_clone/b027f6d2ab84e31a2f4bb10a377a7d43/recovered.edits/4.seqid 2023-07-16 21:15:47,920 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/default/Group_testCloneSnapshot_clone/b027f6d2ab84e31a2f4bb10a377a7d43 2023-07-16 21:15:47,920 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived Group_testCloneSnapshot_clone regions 2023-07-16 21:15:47,923 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=95, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-16 21:15:47,924 WARN [PEWorker-3] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of Group_testCloneSnapshot_clone from hbase:meta 2023-07-16 21:15:47,926 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(421): Removing 'Group_testCloneSnapshot_clone' descriptor. 2023-07-16 21:15:47,927 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=95, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-16 21:15:47,927 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(411): Removing 'Group_testCloneSnapshot_clone' from region states. 2023-07-16 21:15:47,927 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542147927"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:47,929 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-16 21:15:47,929 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b027f6d2ab84e31a2f4bb10a377a7d43, NAME => 'Group_testCloneSnapshot_clone,,1689542145077.b027f6d2ab84e31a2f4bb10a377a7d43.', STARTKEY => '', ENDKEY => ''}] 2023-07-16 21:15:47,929 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(415): Marking 'Group_testCloneSnapshot_clone' as deleted. 2023-07-16 21:15:47,929 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCloneSnapshot_clone","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689542147929"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:47,933 INFO [PEWorker-3] hbase.MetaTableAccessor(1658): Deleted table Group_testCloneSnapshot_clone state from META 2023-07-16 21:15:47,934 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(130): Finished pid=95, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-16 21:15:47,936 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; DeleteTableProcedure table=Group_testCloneSnapshot_clone in 30 msec 2023-07-16 21:15:48,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=95 2023-07-16 21:15:48,013 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:Group_testCloneSnapshot_clone, procId: 95 completed 2023-07-16 21:15:48,016 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:48,016 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:48,017 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:48,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:48,017 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:48,018 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:48,018 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:48,019 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:48,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:48,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:48,024 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:48,027 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:15:48,028 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:48,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:48,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:48,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:48,034 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:48,037 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:48,037 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:48,039 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:48,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:48,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.CallRunner(144): callId: 564 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:39404 deadline: 1689543348039, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:48,040 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:48,041 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:48,042 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:48,042 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:48,042 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:48,043 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:48,043 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:48,062 INFO [Listener at localhost/43605] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testCloneSnapshot Thread=510 (was 506) Potentially hanging thread: (jenkins-hbase4.apache.org,41499,1689542132027)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,33511,1689542134438' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,41213,1689542138767' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-301091760_17 at /127.0.0.1:45048 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,36993,1689542134044' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-870215550_17 at /127.0.0.1:55850 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,45229,1689542134233' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x50aa0278-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-301091760_17 at /127.0.0.1:39154 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=771 (was 771), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=311 (was 311), ProcessCount=172 (was 172), AvailableMemoryMB=2654 (was 2774) 2023-07-16 21:15:48,062 WARN [Listener at localhost/43605] hbase.ResourceChecker(130): Thread=510 is superior to 500 2023-07-16 21:15:48,080 INFO [Listener at localhost/43605] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testCreateWhenRsgroupNoOnlineServers Thread=510, OpenFileDescriptor=771, MaxFileDescriptor=60000, SystemLoadAverage=311, ProcessCount=172, AvailableMemoryMB=2652 2023-07-16 21:15:48,080 WARN [Listener at localhost/43605] hbase.ResourceChecker(130): Thread=510 is superior to 500 2023-07-16 21:15:48,080 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(132): testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:48,085 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:48,085 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:48,086 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:48,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:48,086 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:48,086 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:48,086 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:48,087 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:48,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:48,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:48,093 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:48,095 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:15:48,096 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:48,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:48,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:48,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:48,101 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:48,104 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:48,104 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:48,106 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:48,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:48,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.CallRunner(144): callId: 592 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:39404 deadline: 1689543348106, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:48,107 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:48,108 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:48,109 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:48,109 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:48,109 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:48,110 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:48,110 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:48,110 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBasics(141): testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:48,111 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:48,111 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:48,111 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup appInfo 2023-07-16 21:15:48,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:48,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:48,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-16 21:15:48,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:15:48,117 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:48,119 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:48,120 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:48,122 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:33511] to rsgroup appInfo 2023-07-16 21:15:48,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:48,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:48,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-16 21:15:48,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:15:48,131 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(286): Moving 0 region(s) to group default, current retry=0 2023-07-16 21:15:48,131 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(261): All regions from [jenkins-hbase4.apache.org,33511,1689542134438] are moved back to default 2023-07-16 21:15:48,131 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(438): Move servers done: default => appInfo 2023-07-16 21:15:48,131 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:48,134 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:48,134 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:48,136 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=appInfo 2023-07-16 21:15:48,136 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:48,143 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/draining 2023-07-16 21:15:48,144 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.ServerManager(636): Server jenkins-hbase4.apache.org,33511,1689542134438 added to draining server list. 2023-07-16 21:15:48,145 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/draining/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:48,145 WARN [zk-event-processor-pool-0] master.ServerManager(632): Server jenkins-hbase4.apache.org,33511,1689542134438 is already in the draining server list.Ignoring request to add it again. 2023-07-16 21:15:48,146 INFO [zk-event-processor-pool-0] master.DrainingServerTracker(92): Draining RS node created, adding to list [jenkins-hbase4.apache.org,33511,1689542134438] 2023-07-16 21:15:48,149 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'Group_ns', hbase.rsgroup.name => 'appInfo'} 2023-07-16 21:15:48,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=Group_ns 2023-07-16 21:15:48,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-07-16 21:15:48,157 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-16 21:15:48,159 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; CreateNamespaceProcedure, namespace=Group_ns in 9 msec 2023-07-16 21:15:48,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-07-16 21:15:48,255 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'Group_ns:testCreateWhenRsgroupNoOnlineServers', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-16 21:15:48,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:48,258 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=97, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers execute state=CREATE_TABLE_PRE_OPERATION 2023-07-16 21:15:48,258 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "Group_ns" qualifier: "testCreateWhenRsgroupNoOnlineServers" procId is: 97 2023-07-16 21:15:48,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-07-16 21:15:48,272 INFO [PEWorker-2] procedure2.ProcedureExecutor(1528): Rolled back pid=97, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-create-table:org.apache.hadoop.hbase.HBaseIOException: No online servers in the rsgroup appInfo which table Group_ns:testCreateWhenRsgroupNoOnlineServers belongs to; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers exec-time=17 msec 2023-07-16 21:15:48,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-07-16 21:15:48,363 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3548): Operation: CREATE, Table Name: Group_ns:testCreateWhenRsgroupNoOnlineServers, procId: 97 failed with No online servers in the rsgroup appInfo which table Group_ns:testCreateWhenRsgroupNoOnlineServers belongs to 2023-07-16 21:15:48,363 DEBUG [Listener at localhost/43605] rsgroup.TestRSGroupsBasics(162): create table error org.apache.hadoop.hbase.HBaseIOException: No online servers in the rsgroup appInfo which table Group_ns:testCreateWhenRsgroupNoOnlineServers belongs to at java.lang.Thread.getStackTrace(Thread.java:1564) at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:130) at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:149) at org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:186) at org.apache.hadoop.hbase.client.Admin.createTable(Admin.java:302) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.testCreateWhenRsgroupNoOnlineServers(TestRSGroupsBasics.java:159) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) at --------Future.get--------(Unknown Source) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.validateRSGroup(RSGroupAdminEndpoint.java:540) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.moveTableToValidRSGroup(RSGroupAdminEndpoint.java:529) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.preCreateTableAction(RSGroupAdminEndpoint.java:501) at org.apache.hadoop.hbase.master.MasterCoprocessorHost$16.call(MasterCoprocessorHost.java:371) at org.apache.hadoop.hbase.master.MasterCoprocessorHost$16.call(MasterCoprocessorHost.java:368) at org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:558) at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:631) at org.apache.hadoop.hbase.master.MasterCoprocessorHost.preCreateTableAction(MasterCoprocessorHost.java:368) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.preCreate(CreateTableProcedure.java:267) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:93) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:53) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-16 21:15:48,370 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/draining/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:48,370 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/draining 2023-07-16 21:15:48,370 INFO [zk-event-processor-pool-0] master.DrainingServerTracker(109): Draining RS node deleted, removing from list [jenkins-hbase4.apache.org,33511,1689542134438] 2023-07-16 21:15:48,373 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'Group_ns:testCreateWhenRsgroupNoOnlineServers', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-16 21:15:48,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=98, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:48,377 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=98, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers execute state=CREATE_TABLE_PRE_OPERATION 2023-07-16 21:15:48,377 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "Group_ns" qualifier: "testCreateWhenRsgroupNoOnlineServers" procId is: 98 2023-07-16 21:15:48,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=98 2023-07-16 21:15:48,379 DEBUG [PEWorker-4] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:48,379 DEBUG [PEWorker-4] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:48,380 DEBUG [PEWorker-4] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-16 21:15:48,380 DEBUG [PEWorker-4] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:15:48,383 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=98, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-16 21:15:48,384 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/dc670cfb9d1fccb62887bb284da4230f 2023-07-16 21:15:48,385 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/dc670cfb9d1fccb62887bb284da4230f empty. 2023-07-16 21:15:48,385 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/dc670cfb9d1fccb62887bb284da4230f 2023-07-16 21:15:48,386 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived Group_ns:testCreateWhenRsgroupNoOnlineServers regions 2023-07-16 21:15:48,401 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/.tabledesc/.tableinfo.0000000001 2023-07-16 21:15:48,403 INFO [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(7675): creating {ENCODED => dc670cfb9d1fccb62887bb284da4230f, NAME => 'Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='Group_ns:testCreateWhenRsgroupNoOnlineServers', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:48,413 DEBUG [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(866): Instantiated Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:48,413 DEBUG [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(1604): Closing dc670cfb9d1fccb62887bb284da4230f, disabling compactions & flushes 2023-07-16 21:15:48,413 INFO [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(1626): Closing region Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f. 2023-07-16 21:15:48,413 DEBUG [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f. 2023-07-16 21:15:48,413 DEBUG [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f. after waiting 0 ms 2023-07-16 21:15:48,413 DEBUG [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f. 2023-07-16 21:15:48,413 INFO [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(1838): Closed Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f. 2023-07-16 21:15:48,413 DEBUG [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(1558): Region close journal for dc670cfb9d1fccb62887bb284da4230f: 2023-07-16 21:15:48,415 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=98, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers execute state=CREATE_TABLE_ADD_TO_META 2023-07-16 21:15:48,416 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542148416"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542148416"}]},"ts":"1689542148416"} 2023-07-16 21:15:48,418 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-16 21:15:48,418 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=98, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-16 21:15:48,418 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542148418"}]},"ts":"1689542148418"} 2023-07-16 21:15:48,420 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=Group_ns:testCreateWhenRsgroupNoOnlineServers, state=ENABLING in hbase:meta 2023-07-16 21:15:48,423 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers, region=dc670cfb9d1fccb62887bb284da4230f, ASSIGN}] 2023-07-16 21:15:48,424 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=98, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers, region=dc670cfb9d1fccb62887bb284da4230f, ASSIGN 2023-07-16 21:15:48,425 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=98, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers, region=dc670cfb9d1fccb62887bb284da4230f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33511,1689542134438; forceNewPlan=false, retain=false 2023-07-16 21:15:48,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=98 2023-07-16 21:15:48,577 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=dc670cfb9d1fccb62887bb284da4230f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:48,577 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542148577"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542148577"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542148577"}]},"ts":"1689542148577"} 2023-07-16 21:15:48,579 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=99, state=RUNNABLE; OpenRegionProcedure dc670cfb9d1fccb62887bb284da4230f, server=jenkins-hbase4.apache.org,33511,1689542134438}] 2023-07-16 21:15:48,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=98 2023-07-16 21:15:48,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f. 2023-07-16 21:15:48,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dc670cfb9d1fccb62887bb284da4230f, NAME => 'Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:15:48,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCreateWhenRsgroupNoOnlineServers dc670cfb9d1fccb62887bb284da4230f 2023-07-16 21:15:48,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:48,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for dc670cfb9d1fccb62887bb284da4230f 2023-07-16 21:15:48,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for dc670cfb9d1fccb62887bb284da4230f 2023-07-16 21:15:48,737 INFO [StoreOpener-dc670cfb9d1fccb62887bb284da4230f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region dc670cfb9d1fccb62887bb284da4230f 2023-07-16 21:15:48,740 DEBUG [StoreOpener-dc670cfb9d1fccb62887bb284da4230f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/dc670cfb9d1fccb62887bb284da4230f/f 2023-07-16 21:15:48,740 DEBUG [StoreOpener-dc670cfb9d1fccb62887bb284da4230f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/dc670cfb9d1fccb62887bb284da4230f/f 2023-07-16 21:15:48,740 INFO [StoreOpener-dc670cfb9d1fccb62887bb284da4230f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dc670cfb9d1fccb62887bb284da4230f columnFamilyName f 2023-07-16 21:15:48,741 INFO [StoreOpener-dc670cfb9d1fccb62887bb284da4230f-1] regionserver.HStore(310): Store=dc670cfb9d1fccb62887bb284da4230f/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:48,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/dc670cfb9d1fccb62887bb284da4230f 2023-07-16 21:15:48,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/dc670cfb9d1fccb62887bb284da4230f 2023-07-16 21:15:48,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for dc670cfb9d1fccb62887bb284da4230f 2023-07-16 21:15:48,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/dc670cfb9d1fccb62887bb284da4230f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:48,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened dc670cfb9d1fccb62887bb284da4230f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11315305760, jitterRate=0.05381996929645538}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:48,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for dc670cfb9d1fccb62887bb284da4230f: 2023-07-16 21:15:48,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f., pid=100, masterSystemTime=1689542148731 2023-07-16 21:15:48,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f. 2023-07-16 21:15:48,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f. 2023-07-16 21:15:48,752 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=dc670cfb9d1fccb62887bb284da4230f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:48,753 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542148752"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542148752"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542148752"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542148752"}]},"ts":"1689542148752"} 2023-07-16 21:15:48,756 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=99 2023-07-16 21:15:48,756 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=99, state=SUCCESS; OpenRegionProcedure dc670cfb9d1fccb62887bb284da4230f, server=jenkins-hbase4.apache.org,33511,1689542134438 in 175 msec 2023-07-16 21:15:48,759 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-07-16 21:15:48,759 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; TransitRegionStateProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers, region=dc670cfb9d1fccb62887bb284da4230f, ASSIGN in 333 msec 2023-07-16 21:15:48,761 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=98, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-16 21:15:48,761 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542148761"}]},"ts":"1689542148761"} 2023-07-16 21:15:48,763 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=Group_ns:testCreateWhenRsgroupNoOnlineServers, state=ENABLED in hbase:meta 2023-07-16 21:15:48,769 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=98, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers execute state=CREATE_TABLE_POST_OPERATION 2023-07-16 21:15:48,770 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=98, state=SUCCESS; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers in 395 msec 2023-07-16 21:15:48,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=98 2023-07-16 21:15:48,981 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: Group_ns:testCreateWhenRsgroupNoOnlineServers, procId: 98 completed 2023-07-16 21:15:48,982 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:48,987 INFO [Listener at localhost/43605] client.HBaseAdmin$15(890): Started disable of Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:48,988 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:48,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:48,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-07-16 21:15:48,992 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542148992"}]},"ts":"1689542148992"} 2023-07-16 21:15:48,993 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=Group_ns:testCreateWhenRsgroupNoOnlineServers, state=DISABLING in hbase:meta 2023-07-16 21:15:48,994 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set Group_ns:testCreateWhenRsgroupNoOnlineServers to state=DISABLING 2023-07-16 21:15:48,995 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=101, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers, region=dc670cfb9d1fccb62887bb284da4230f, UNASSIGN}] 2023-07-16 21:15:48,998 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=102, ppid=101, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers, region=dc670cfb9d1fccb62887bb284da4230f, UNASSIGN 2023-07-16 21:15:48,999 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=102 updating hbase:meta row=dc670cfb9d1fccb62887bb284da4230f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:48,999 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542148999"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542148999"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542148999"}]},"ts":"1689542148999"} 2023-07-16 21:15:49,000 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE; CloseRegionProcedure dc670cfb9d1fccb62887bb284da4230f, server=jenkins-hbase4.apache.org,33511,1689542134438}] 2023-07-16 21:15:49,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-07-16 21:15:49,153 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close dc670cfb9d1fccb62887bb284da4230f 2023-07-16 21:15:49,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing dc670cfb9d1fccb62887bb284da4230f, disabling compactions & flushes 2023-07-16 21:15:49,154 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f. 2023-07-16 21:15:49,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f. 2023-07-16 21:15:49,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f. after waiting 0 ms 2023-07-16 21:15:49,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f. 2023-07-16 21:15:49,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/dc670cfb9d1fccb62887bb284da4230f/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:15:49,159 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f. 2023-07-16 21:15:49,159 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for dc670cfb9d1fccb62887bb284da4230f: 2023-07-16 21:15:49,161 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed dc670cfb9d1fccb62887bb284da4230f 2023-07-16 21:15:49,161 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=102 updating hbase:meta row=dc670cfb9d1fccb62887bb284da4230f, regionState=CLOSED 2023-07-16 21:15:49,161 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1689542149161"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542149161"}]},"ts":"1689542149161"} 2023-07-16 21:15:49,164 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-07-16 21:15:49,164 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; CloseRegionProcedure dc670cfb9d1fccb62887bb284da4230f, server=jenkins-hbase4.apache.org,33511,1689542134438 in 163 msec 2023-07-16 21:15:49,166 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=101 2023-07-16 21:15:49,166 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=101, state=SUCCESS; TransitRegionStateProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers, region=dc670cfb9d1fccb62887bb284da4230f, UNASSIGN in 169 msec 2023-07-16 21:15:49,167 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542149167"}]},"ts":"1689542149167"} 2023-07-16 21:15:49,168 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=Group_ns:testCreateWhenRsgroupNoOnlineServers, state=DISABLED in hbase:meta 2023-07-16 21:15:49,169 INFO [PEWorker-3] procedure.DisableTableProcedure(305): Set Group_ns:testCreateWhenRsgroupNoOnlineServers to state=DISABLED 2023-07-16 21:15:49,171 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DisableTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers in 182 msec 2023-07-16 21:15:49,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-07-16 21:15:49,294 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: Group_ns:testCreateWhenRsgroupNoOnlineServers, procId: 101 completed 2023-07-16 21:15:49,294 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:49,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:49,297 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=104, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:49,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint(577): Removing deleted table 'Group_ns:testCreateWhenRsgroupNoOnlineServers' from rsgroup 'appInfo' 2023-07-16 21:15:49,298 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=104, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:49,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:49,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:49,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-16 21:15:49,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:15:49,303 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/dc670cfb9d1fccb62887bb284da4230f 2023-07-16 21:15:49,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=104 2023-07-16 21:15:49,305 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/dc670cfb9d1fccb62887bb284da4230f/f, FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/dc670cfb9d1fccb62887bb284da4230f/recovered.edits] 2023-07-16 21:15:49,312 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/dc670cfb9d1fccb62887bb284da4230f/recovered.edits/4.seqid to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/dc670cfb9d1fccb62887bb284da4230f/recovered.edits/4.seqid 2023-07-16 21:15:49,312 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/dc670cfb9d1fccb62887bb284da4230f 2023-07-16 21:15:49,312 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived Group_ns:testCreateWhenRsgroupNoOnlineServers regions 2023-07-16 21:15:49,315 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=104, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:49,317 WARN [PEWorker-5] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of Group_ns:testCreateWhenRsgroupNoOnlineServers from hbase:meta 2023-07-16 21:15:49,318 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(421): Removing 'Group_ns:testCreateWhenRsgroupNoOnlineServers' descriptor. 2023-07-16 21:15:49,319 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=104, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:49,319 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(411): Removing 'Group_ns:testCreateWhenRsgroupNoOnlineServers' from region states. 2023-07-16 21:15:49,319 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689542149319"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:49,321 INFO [PEWorker-5] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-16 21:15:49,321 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => dc670cfb9d1fccb62887bb284da4230f, NAME => 'Group_ns:testCreateWhenRsgroupNoOnlineServers,,1689542148373.dc670cfb9d1fccb62887bb284da4230f.', STARTKEY => '', ENDKEY => ''}] 2023-07-16 21:15:49,321 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(415): Marking 'Group_ns:testCreateWhenRsgroupNoOnlineServers' as deleted. 2023-07-16 21:15:49,321 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689542149321"}]},"ts":"9223372036854775807"} 2023-07-16 21:15:49,323 INFO [PEWorker-5] hbase.MetaTableAccessor(1658): Deleted table Group_ns:testCreateWhenRsgroupNoOnlineServers state from META 2023-07-16 21:15:49,327 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(130): Finished pid=104, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:49,328 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers in 33 msec 2023-07-16 21:15:49,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=104 2023-07-16 21:15:49,404 INFO [Listener at localhost/43605] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: Group_ns:testCreateWhenRsgroupNoOnlineServers, procId: 104 completed 2023-07-16 21:15:49,408 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.HMaster$17(3086): Client=jenkins//172.31.14.131 delete Group_ns 2023-07-16 21:15:49,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DELETE_NAMESPACE_PREPARE; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-16 21:15:49,410 INFO [PEWorker-2] procedure.DeleteNamespaceProcedure(73): pid=105, state=RUNNABLE:DELETE_NAMESPACE_PREPARE, locked=true; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-16 21:15:49,413 INFO [PEWorker-2] procedure.DeleteNamespaceProcedure(73): pid=105, state=RUNNABLE:DELETE_NAMESPACE_DELETE_FROM_NS_TABLE, locked=true; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-16 21:15:49,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-07-16 21:15:49,414 INFO [PEWorker-2] procedure.DeleteNamespaceProcedure(73): pid=105, state=RUNNABLE:DELETE_NAMESPACE_REMOVE_FROM_ZK, locked=true; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-16 21:15:49,416 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/namespace/Group_ns 2023-07-16 21:15:49,416 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-16 21:15:49,416 INFO [PEWorker-2] procedure.DeleteNamespaceProcedure(73): pid=105, state=RUNNABLE:DELETE_NAMESPACE_DELETE_DIRECTORIES, locked=true; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-16 21:15:49,418 INFO [PEWorker-2] procedure.DeleteNamespaceProcedure(73): pid=105, state=RUNNABLE:DELETE_NAMESPACE_REMOVE_NAMESPACE_QUOTA, locked=true; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-16 21:15:49,419 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DeleteNamespaceProcedure, namespace=Group_ns in 10 msec 2023-07-16 21:15:49,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-07-16 21:15:49,515 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:49,515 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:49,516 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:49,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:49,516 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:49,517 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:49,517 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:49,518 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:49,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:49,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-16 21:15:49,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 5 2023-07-16 21:15:49,523 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:49,524 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:49,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:49,524 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:49,525 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:33511] to rsgroup default 2023-07-16 21:15:49,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:49,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-16 21:15:49,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:49,533 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(286): Moving 0 region(s) to group appInfo, current retry=0 2023-07-16 21:15:49,533 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(261): All regions from [jenkins-hbase4.apache.org,33511,1689542134438] are moved back to appInfo 2023-07-16 21:15:49,533 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(438): Move servers done: appInfo => default 2023-07-16 21:15:49,533 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:49,534 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup appInfo 2023-07-16 21:15:49,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:49,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:49,539 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:49,541 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:15:49,542 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:49,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:49,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:49,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:49,547 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:49,550 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:49,550 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:49,552 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:49,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:49,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.CallRunner(144): callId: 694 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:39404 deadline: 1689543349552, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:49,552 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:49,554 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:49,555 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:49,555 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:49,556 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:49,556 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:49,556 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:49,575 INFO [Listener at localhost/43605] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testCreateWhenRsgroupNoOnlineServers Thread=513 (was 510) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-870215550_17 at /127.0.0.1:39154 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=769 (was 771), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=311 (was 311), ProcessCount=170 (was 172), AvailableMemoryMB=4879 (was 2652) - AvailableMemoryMB LEAK? - 2023-07-16 21:15:49,575 WARN [Listener at localhost/43605] hbase.ResourceChecker(130): Thread=513 is superior to 500 2023-07-16 21:15:49,593 INFO [Listener at localhost/43605] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testBasicStartUp Thread=513, OpenFileDescriptor=769, MaxFileDescriptor=60000, SystemLoadAverage=311, ProcessCount=170, AvailableMemoryMB=4879 2023-07-16 21:15:49,593 WARN [Listener at localhost/43605] hbase.ResourceChecker(130): Thread=513 is superior to 500 2023-07-16 21:15:49,593 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(132): testBasicStartUp 2023-07-16 21:15:49,597 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:49,597 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:49,598 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:49,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:49,598 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:49,599 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:49,599 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:49,600 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:49,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:49,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:49,605 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:49,607 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:15:49,608 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:49,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:49,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:49,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:49,613 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:49,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:49,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:49,618 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:49,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:49,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.CallRunner(144): callId: 722 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:39404 deadline: 1689543349618, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:49,618 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:49,620 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:49,621 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:49,621 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:49,621 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:49,622 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:49,622 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:49,622 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:49,623 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:49,626 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:49,626 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:49,627 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:49,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:49,627 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:49,627 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:49,627 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:49,628 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:49,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:49,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:49,639 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:49,641 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:15:49,642 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:49,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:49,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:49,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:49,646 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:49,648 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:49,648 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:49,650 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:49,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:49,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.CallRunner(144): callId: 752 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:39404 deadline: 1689543349650, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:49,651 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:49,653 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:49,653 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:49,653 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:49,653 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:49,654 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:49,654 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:49,671 INFO [Listener at localhost/43605] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testBasicStartUp Thread=514 (was 513) Potentially hanging thread: hconnection-0x609dbbf-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=769 (was 769), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=311 (was 311), ProcessCount=170 (was 170), AvailableMemoryMB=4879 (was 4879) 2023-07-16 21:15:49,671 WARN [Listener at localhost/43605] hbase.ResourceChecker(130): Thread=514 is superior to 500 2023-07-16 21:15:49,686 INFO [Listener at localhost/43605] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testRSGroupsWithHBaseQuota Thread=514, OpenFileDescriptor=769, MaxFileDescriptor=60000, SystemLoadAverage=311, ProcessCount=170, AvailableMemoryMB=4879 2023-07-16 21:15:49,687 WARN [Listener at localhost/43605] hbase.ResourceChecker(130): Thread=514 is superior to 500 2023-07-16 21:15:49,687 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(132): testRSGroupsWithHBaseQuota 2023-07-16 21:15:49,691 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:49,691 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:49,692 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:15:49,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:15:49,692 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:15:49,693 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:15:49,693 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:15:49,694 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:15:49,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:49,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:15:49,700 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:15:49,703 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:15:49,704 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:15:49,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:15:49,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:15:49,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:15:49,715 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:15:49,717 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:49,717 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:49,720 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41499] to rsgroup master 2023-07-16 21:15:49,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:15:49,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] ipc.CallRunner(144): callId: 780 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:39404 deadline: 1689543349719, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. 2023-07-16 21:15:49,720 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:41499 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:15:49,722 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:49,723 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:15:49,723 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:15:49,723 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33511, jenkins-hbase4.apache.org:36993, jenkins-hbase4.apache.org:41213, jenkins-hbase4.apache.org:45229], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:15:49,724 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:15:49,724 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41499] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:15:49,724 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBasics(309): Shutting down cluster 2023-07-16 21:15:49,725 INFO [Listener at localhost/43605] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-16 21:15:49,725 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7c41c5f7 to 127.0.0.1:63917 2023-07-16 21:15:49,725 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:15:49,726 DEBUG [Listener at localhost/43605] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-16 21:15:49,726 DEBUG [Listener at localhost/43605] util.JVMClusterUtil(257): Found active master hash=1007221288, stopped=false 2023-07-16 21:15:49,726 DEBUG [Listener at localhost/43605] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint 2023-07-16 21:15:49,727 DEBUG [Listener at localhost/43605] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver 2023-07-16 21:15:49,727 INFO [Listener at localhost/43605] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,41499,1689542132027 2023-07-16 21:15:49,729 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-16 21:15:49,729 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-16 21:15:49,729 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-16 21:15:49,729 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:15:49,729 INFO [Listener at localhost/43605] procedure2.ProcedureExecutor(629): Stopping 2023-07-16 21:15:49,729 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-16 21:15:49,729 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-16 21:15:49,730 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:15:49,730 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:15:49,730 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:15:49,730 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:15:49,730 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x084e601d to 127.0.0.1:63917 2023-07-16 21:15:49,730 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:15:49,731 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:15:49,731 INFO [Listener at localhost/43605] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36993,1689542134044' ***** 2023-07-16 21:15:49,731 INFO [Listener at localhost/43605] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-16 21:15:49,731 INFO [Listener at localhost/43605] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45229,1689542134233' ***** 2023-07-16 21:15:49,731 INFO [Listener at localhost/43605] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-16 21:15:49,731 INFO [Listener at localhost/43605] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33511,1689542134438' ***** 2023-07-16 21:15:49,732 INFO [Listener at localhost/43605] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-16 21:15:49,731 INFO [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:15:49,732 INFO [Listener at localhost/43605] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41213,1689542138767' ***** 2023-07-16 21:15:49,732 INFO [Listener at localhost/43605] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-16 21:15:49,732 INFO [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:15:49,731 INFO [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:15:49,733 INFO [RS:4;jenkins-hbase4:41213] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:15:49,746 INFO [RS:0;jenkins-hbase4:36993] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@5a679f9e{regionserver,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:15:49,746 INFO [RS:4;jenkins-hbase4:41213] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@1899cb6d{regionserver,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:15:49,746 INFO [RS:2;jenkins-hbase4:33511] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@4ad0acc{regionserver,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:15:49,746 INFO [RS:1;jenkins-hbase4:45229] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@7c8e2ea{regionserver,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:15:49,746 INFO [RS:0;jenkins-hbase4:36993] server.AbstractConnector(383): Stopped ServerConnector@25eab107{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:15:49,746 INFO [RS:0;jenkins-hbase4:36993] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:15:49,746 INFO [RS:2;jenkins-hbase4:33511] server.AbstractConnector(383): Stopped ServerConnector@5a58295b{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:15:49,746 INFO [RS:4;jenkins-hbase4:41213] server.AbstractConnector(383): Stopped ServerConnector@9b30d54{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:15:49,747 INFO [RS:2;jenkins-hbase4:33511] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:15:49,747 INFO [RS:0;jenkins-hbase4:36993] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@7ea3cad5{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:15:49,747 INFO [RS:4;jenkins-hbase4:41213] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:15:49,749 INFO [RS:0;jenkins-hbase4:36993] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@69738833{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:15:49,747 INFO [RS:1;jenkins-hbase4:45229] server.AbstractConnector(383): Stopped ServerConnector@12e45025{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:15:49,749 INFO [RS:4;jenkins-hbase4:41213] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@67ca648{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:15:49,749 INFO [RS:2;jenkins-hbase4:33511] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@6ae3c253{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:15:49,750 INFO [RS:0;jenkins-hbase4:36993] regionserver.HeapMemoryManager(220): Stopping 2023-07-16 21:15:49,750 INFO [RS:4;jenkins-hbase4:41213] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@78bafdff{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:15:49,749 INFO [RS:1;jenkins-hbase4:45229] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:15:49,751 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-16 21:15:49,751 INFO [RS:0;jenkins-hbase4:36993] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-16 21:15:49,751 INFO [RS:2;jenkins-hbase4:33511] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@50d1e028{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:15:49,752 INFO [RS:4;jenkins-hbase4:41213] regionserver.HeapMemoryManager(220): Stopping 2023-07-16 21:15:49,752 INFO [RS:0;jenkins-hbase4:36993] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-16 21:15:49,752 INFO [RS:1;jenkins-hbase4:45229] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@38834d1b{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:15:49,752 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-16 21:15:49,752 INFO [RS:4;jenkins-hbase4:41213] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-16 21:15:49,753 INFO [RS:1;jenkins-hbase4:45229] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@425bb8be{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:15:49,753 INFO [RS:4;jenkins-hbase4:41213] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-16 21:15:49,753 INFO [RS:4;jenkins-hbase4:41213] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:49,753 DEBUG [RS:4;jenkins-hbase4:41213] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1f4f67ca to 127.0.0.1:63917 2023-07-16 21:15:49,754 DEBUG [RS:4;jenkins-hbase4:41213] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:15:49,754 INFO [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:49,754 DEBUG [RS:0;jenkins-hbase4:36993] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x285bc483 to 127.0.0.1:63917 2023-07-16 21:15:49,754 DEBUG [RS:0;jenkins-hbase4:36993] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:15:49,754 INFO [RS:0;jenkins-hbase4:36993] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-16 21:15:49,754 INFO [RS:0;jenkins-hbase4:36993] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-16 21:15:49,754 INFO [RS:0;jenkins-hbase4:36993] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-16 21:15:49,754 INFO [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-16 21:15:49,754 INFO [RS:2;jenkins-hbase4:33511] regionserver.HeapMemoryManager(220): Stopping 2023-07-16 21:15:49,754 INFO [RS:4;jenkins-hbase4:41213] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41213,1689542138767; all regions closed. 2023-07-16 21:15:49,754 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-16 21:15:49,754 INFO [RS:2;jenkins-hbase4:33511] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-16 21:15:49,754 INFO [RS:1;jenkins-hbase4:45229] regionserver.HeapMemoryManager(220): Stopping 2023-07-16 21:15:49,755 INFO [RS:2;jenkins-hbase4:33511] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-16 21:15:49,755 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-16 21:15:49,755 INFO [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:49,755 INFO [RS:1;jenkins-hbase4:45229] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-16 21:15:49,755 DEBUG [RS:2;jenkins-hbase4:33511] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x783ea80a to 127.0.0.1:63917 2023-07-16 21:15:49,755 DEBUG [RS:2;jenkins-hbase4:33511] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:15:49,755 INFO [RS:1;jenkins-hbase4:45229] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-16 21:15:49,755 INFO [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33511,1689542134438; all regions closed. 2023-07-16 21:15:49,755 INFO [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(3305): Received CLOSE for 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:49,756 INFO [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(3305): Received CLOSE for 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:49,756 INFO [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:49,757 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 73c0efa65c224047ddc5ff1870c16b5e, disabling compactions & flushes 2023-07-16 21:15:49,757 DEBUG [RS:1;jenkins-hbase4:45229] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6710a5a7 to 127.0.0.1:63917 2023-07-16 21:15:49,757 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:15:49,757 INFO [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-16 21:15:49,757 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:15:49,757 DEBUG [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-07-16 21:15:49,757 DEBUG [RS:1;jenkins-hbase4:45229] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:15:49,757 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. after waiting 0 ms 2023-07-16 21:15:49,758 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-16 21:15:49,757 INFO [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-16 21:15:49,758 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-16 21:15:49,758 DEBUG [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1478): Online Regions={73c0efa65c224047ddc5ff1870c16b5e=hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e., 82d35ab01095f590b40a2bc17e2059db=hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.} 2023-07-16 21:15:49,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:15:49,758 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-16 21:15:49,758 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-16 21:15:49,758 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-16 21:15:49,758 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 73c0efa65c224047ddc5ff1870c16b5e 1/1 column families, dataSize=365 B heapSize=1.13 KB 2023-07-16 21:15:49,758 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=45.20 KB heapSize=72.77 KB 2023-07-16 21:15:49,758 DEBUG [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1504): Waiting on 73c0efa65c224047ddc5ff1870c16b5e, 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:49,758 DEBUG [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-16 21:15:49,768 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:15:49,768 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:15:49,768 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:15:49,768 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:15:49,775 DEBUG [RS:4;jenkins-hbase4:41213] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs 2023-07-16 21:15:49,775 INFO [RS:4;jenkins-hbase4:41213] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41213%2C1689542138767:(num 1689542139121) 2023-07-16 21:15:49,775 DEBUG [RS:4;jenkins-hbase4:41213] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:15:49,775 INFO [RS:4;jenkins-hbase4:41213] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:15:49,775 DEBUG [RS:2;jenkins-hbase4:33511] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs 2023-07-16 21:15:49,776 INFO [RS:2;jenkins-hbase4:33511] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33511%2C1689542134438:(num 1689542136569) 2023-07-16 21:15:49,776 DEBUG [RS:2;jenkins-hbase4:33511] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:15:49,776 INFO [RS:2;jenkins-hbase4:33511] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:15:49,776 INFO [RS:4;jenkins-hbase4:41213] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-16 21:15:49,776 INFO [RS:2;jenkins-hbase4:33511] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-16 21:15:49,777 INFO [RS:4;jenkins-hbase4:41213] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-16 21:15:49,777 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:15:49,777 INFO [RS:4;jenkins-hbase4:41213] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-16 21:15:49,777 INFO [RS:2;jenkins-hbase4:33511] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-16 21:15:49,777 INFO [RS:2;jenkins-hbase4:33511] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-16 21:15:49,777 INFO [RS:2;jenkins-hbase4:33511] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-16 21:15:49,777 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:15:49,778 INFO [RS:2;jenkins-hbase4:33511] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33511 2023-07-16 21:15:49,777 INFO [RS:4;jenkins-hbase4:41213] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-16 21:15:49,779 INFO [RS:4;jenkins-hbase4:41213] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41213 2023-07-16 21:15:49,859 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:49,859 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:49,859 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:49,859 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:49,859 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:49,859 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41213,1689542138767 2023-07-16 21:15:49,859 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:49,859 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:49,859 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:49,859 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:49,859 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:49,859 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33511,1689542134438 2023-07-16 21:15:49,859 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:49,958 DEBUG [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1504): Waiting on 73c0efa65c224047ddc5ff1870c16b5e, 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:49,958 DEBUG [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-16 21:15:49,962 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41213,1689542138767] 2023-07-16 21:15:49,962 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41213,1689542138767; numProcessing=1 2023-07-16 21:15:49,963 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41213,1689542138767 already deleted, retry=false 2023-07-16 21:15:49,963 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41213,1689542138767 expired; onlineServers=3 2023-07-16 21:15:49,963 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33511,1689542134438] 2023-07-16 21:15:49,963 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33511,1689542134438; numProcessing=2 2023-07-16 21:15:50,062 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:15:50,062 INFO [RS:2;jenkins-hbase4:33511] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33511,1689542134438; zookeeper connection closed. 2023-07-16 21:15:50,062 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:33511-0x1016ffd80780003, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:15:50,063 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@21338f0a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@21338f0a 2023-07-16 21:15:50,065 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,33511,1689542134438 already deleted, retry=false 2023-07-16 21:15:50,066 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33511,1689542134438 expired; onlineServers=2 2023-07-16 21:15:50,127 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:15:50,127 INFO [RS:4;jenkins-hbase4:41213] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41213,1689542138767; zookeeper connection closed. 2023-07-16 21:15:50,127 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41213-0x1016ffd8078000d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:15:50,128 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3a261585] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3a261585 2023-07-16 21:15:50,159 DEBUG [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1504): Waiting on 73c0efa65c224047ddc5ff1870c16b5e, 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:50,159 DEBUG [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-16 21:15:50,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=365 B at sequenceid=11 (bloomFilter=true), to=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/.tmp/info/e7aca6aa2367450cb461f1ac0a602f75 2023-07-16 21:15:50,260 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=39.56 KB at sequenceid=134 (bloomFilter=false), to=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp/info/8c49957a900a44abbf01b64d8b340ddd 2023-07-16 21:15:50,292 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7aca6aa2367450cb461f1ac0a602f75 2023-07-16 21:15:50,292 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c49957a900a44abbf01b64d8b340ddd 2023-07-16 21:15:50,308 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/.tmp/info/e7aca6aa2367450cb461f1ac0a602f75 as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/info/e7aca6aa2367450cb461f1ac0a602f75 2023-07-16 21:15:50,326 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.73 KB at sequenceid=134 (bloomFilter=false), to=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp/rep_barrier/7a387d081595450199b46921f8b69aeb 2023-07-16 21:15:50,332 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7aca6aa2367450cb461f1ac0a602f75 2023-07-16 21:15:50,332 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/info/e7aca6aa2367450cb461f1ac0a602f75, entries=5, sequenceid=11, filesize=5.1 K 2023-07-16 21:15:50,333 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a387d081595450199b46921f8b69aeb 2023-07-16 21:15:50,334 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~365 B/365, heapSize ~1.11 KB/1136, currentSize=0 B/0 for 73c0efa65c224047ddc5ff1870c16b5e in 576ms, sequenceid=11, compaction requested=false 2023-07-16 21:15:50,341 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/recovered.edits/14.seqid, newMaxSeqId=14, maxSeqId=1 2023-07-16 21:15:50,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:15:50,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 73c0efa65c224047ddc5ff1870c16b5e: 2023-07-16 21:15:50,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:15:50,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 82d35ab01095f590b40a2bc17e2059db, disabling compactions & flushes 2023-07-16 21:15:50,343 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:50,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:50,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. after waiting 0 ms 2023-07-16 21:15:50,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:50,343 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 82d35ab01095f590b40a2bc17e2059db 1/1 column families, dataSize=16.53 KB heapSize=26.77 KB 2023-07-16 21:15:50,353 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.91 KB at sequenceid=134 (bloomFilter=false), to=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp/table/e6b7923a5b4640e084b40f28aa6202c9 2023-07-16 21:15:50,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=16.53 KB at sequenceid=67 (bloomFilter=true), to=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/.tmp/m/d185255a3a8e4435a58d7c4f27f39d36 2023-07-16 21:15:50,359 DEBUG [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1504): Waiting on 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:50,359 DEBUG [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-16 21:15:50,360 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e6b7923a5b4640e084b40f28aa6202c9 2023-07-16 21:15:50,360 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp/info/8c49957a900a44abbf01b64d8b340ddd as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/8c49957a900a44abbf01b64d8b340ddd 2023-07-16 21:15:50,363 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d185255a3a8e4435a58d7c4f27f39d36 2023-07-16 21:15:50,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/.tmp/m/d185255a3a8e4435a58d7c4f27f39d36 as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/d185255a3a8e4435a58d7c4f27f39d36 2023-07-16 21:15:50,367 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c49957a900a44abbf01b64d8b340ddd 2023-07-16 21:15:50,367 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/8c49957a900a44abbf01b64d8b340ddd, entries=52, sequenceid=134, filesize=10.5 K 2023-07-16 21:15:50,369 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp/rep_barrier/7a387d081595450199b46921f8b69aeb as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/rep_barrier/7a387d081595450199b46921f8b69aeb 2023-07-16 21:15:50,372 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d185255a3a8e4435a58d7c4f27f39d36 2023-07-16 21:15:50,372 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/d185255a3a8e4435a58d7c4f27f39d36, entries=21, sequenceid=67, filesize=5.7 K 2023-07-16 21:15:50,373 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~16.53 KB/16928, heapSize ~26.76 KB/27400, currentSize=0 B/0 for 82d35ab01095f590b40a2bc17e2059db in 30ms, sequenceid=67, compaction requested=false 2023-07-16 21:15:50,377 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a387d081595450199b46921f8b69aeb 2023-07-16 21:15:50,377 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/rep_barrier/7a387d081595450199b46921f8b69aeb, entries=16, sequenceid=134, filesize=6.7 K 2023-07-16 21:15:50,378 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp/table/e6b7923a5b4640e084b40f28aa6202c9 as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table/e6b7923a5b4640e084b40f28aa6202c9 2023-07-16 21:15:50,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/recovered.edits/70.seqid, newMaxSeqId=70, maxSeqId=1 2023-07-16 21:15:50,384 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-16 21:15:50,384 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:50,384 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 82d35ab01095f590b40a2bc17e2059db: 2023-07-16 21:15:50,385 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:50,386 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e6b7923a5b4640e084b40f28aa6202c9 2023-07-16 21:15:50,386 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table/e6b7923a5b4640e084b40f28aa6202c9, entries=27, sequenceid=134, filesize=7.1 K 2023-07-16 21:15:50,387 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~45.20 KB/46284, heapSize ~72.73 KB/74472, currentSize=0 B/0 for 1588230740 in 629ms, sequenceid=134, compaction requested=false 2023-07-16 21:15:50,401 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-16 21:15:50,401 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-16 21:15:50,402 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-16 21:15:50,403 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-16 21:15:50,427 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/recovered.edits/137.seqid, newMaxSeqId=137, maxSeqId=1 2023-07-16 21:15:50,428 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-16 21:15:50,428 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-16 21:15:50,428 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-16 21:15:50,428 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-16 21:15:50,560 INFO [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45229,1689542134233; all regions closed. 2023-07-16 21:15:50,561 INFO [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36993,1689542134044; all regions closed. 2023-07-16 21:15:50,574 DEBUG [RS:1;jenkins-hbase4:45229] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs 2023-07-16 21:15:50,574 INFO [RS:1;jenkins-hbase4:45229] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45229%2C1689542134233:(num 1689542136569) 2023-07-16 21:15:50,575 DEBUG [RS:1;jenkins-hbase4:45229] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:15:50,575 INFO [RS:1;jenkins-hbase4:45229] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:15:50,579 INFO [RS:1;jenkins-hbase4:45229] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-16 21:15:50,580 INFO [RS:1;jenkins-hbase4:45229] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-16 21:15:50,580 INFO [RS:1;jenkins-hbase4:45229] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-16 21:15:50,580 INFO [RS:1;jenkins-hbase4:45229] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-16 21:15:50,580 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:15:50,581 INFO [RS:1;jenkins-hbase4:45229] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45229 2023-07-16 21:15:50,583 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:50,583 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,45229,1689542134233 2023-07-16 21:15:50,583 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:50,588 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45229,1689542134233] 2023-07-16 21:15:50,588 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45229,1689542134233; numProcessing=3 2023-07-16 21:15:50,594 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,45229,1689542134233 already deleted, retry=false 2023-07-16 21:15:50,594 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45229,1689542134233 expired; onlineServers=1 2023-07-16 21:15:50,603 DEBUG [RS:0;jenkins-hbase4:36993] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs 2023-07-16 21:15:50,603 INFO [RS:0;jenkins-hbase4:36993] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36993%2C1689542134044.meta:.meta(num 1689542136753) 2023-07-16 21:15:50,611 DEBUG [RS:0;jenkins-hbase4:36993] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs 2023-07-16 21:15:50,611 INFO [RS:0;jenkins-hbase4:36993] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36993%2C1689542134044:(num 1689542136569) 2023-07-16 21:15:50,612 DEBUG [RS:0;jenkins-hbase4:36993] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:15:50,612 INFO [RS:0;jenkins-hbase4:36993] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:15:50,612 INFO [RS:0;jenkins-hbase4:36993] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-16 21:15:50,612 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:15:50,615 INFO [RS:0;jenkins-hbase4:36993] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36993 2023-07-16 21:15:50,618 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:50,618 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36993,1689542134044 2023-07-16 21:15:50,620 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36993,1689542134044] 2023-07-16 21:15:50,620 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36993,1689542134044; numProcessing=4 2023-07-16 21:15:50,621 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36993,1689542134044 already deleted, retry=false 2023-07-16 21:15:50,621 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36993,1689542134044 expired; onlineServers=0 2023-07-16 21:15:50,621 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41499,1689542132027' ***** 2023-07-16 21:15:50,621 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-16 21:15:50,622 DEBUG [M:0;jenkins-hbase4:41499] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7d35f013, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:15:50,622 INFO [M:0;jenkins-hbase4:41499] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:15:50,624 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-07-16 21:15:50,624 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:15:50,624 INFO [M:0;jenkins-hbase4:41499] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@78cf9e5c{master,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/master} 2023-07-16 21:15:50,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-16 21:15:50,625 INFO [M:0;jenkins-hbase4:41499] server.AbstractConnector(383): Stopped ServerConnector@5ce4cc97{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:15:50,625 INFO [M:0;jenkins-hbase4:41499] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:15:50,625 INFO [M:0;jenkins-hbase4:41499] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@2356c1aa{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:15:50,626 INFO [M:0;jenkins-hbase4:41499] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@687841de{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:15:50,626 INFO [M:0;jenkins-hbase4:41499] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41499,1689542132027 2023-07-16 21:15:50,626 INFO [M:0;jenkins-hbase4:41499] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41499,1689542132027; all regions closed. 2023-07-16 21:15:50,626 DEBUG [M:0;jenkins-hbase4:41499] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:15:50,626 INFO [M:0;jenkins-hbase4:41499] master.HMaster(1491): Stopping master jetty server 2023-07-16 21:15:50,627 INFO [M:0;jenkins-hbase4:41499] server.AbstractConnector(383): Stopped ServerConnector@30d25553{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:15:50,627 DEBUG [M:0;jenkins-hbase4:41499] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-16 21:15:50,627 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-16 21:15:50,627 DEBUG [M:0;jenkins-hbase4:41499] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-16 21:15:50,627 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689542136162] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689542136162,5,FailOnTimeoutGroup] 2023-07-16 21:15:50,628 INFO [M:0;jenkins-hbase4:41499] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-16 21:15:50,628 INFO [M:0;jenkins-hbase4:41499] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-16 21:15:50,628 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689542136158] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689542136158,5,FailOnTimeoutGroup] 2023-07-16 21:15:50,628 INFO [M:0;jenkins-hbase4:41499] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-16 21:15:50,628 DEBUG [M:0;jenkins-hbase4:41499] master.HMaster(1512): Stopping service threads 2023-07-16 21:15:50,628 INFO [M:0;jenkins-hbase4:41499] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-16 21:15:50,628 ERROR [M:0;jenkins-hbase4:41499] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-07-16 21:15:50,629 INFO [M:0;jenkins-hbase4:41499] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-16 21:15:50,629 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-16 21:15:50,629 DEBUG [M:0;jenkins-hbase4:41499] zookeeper.ZKUtil(398): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-07-16 21:15:50,629 WARN [M:0;jenkins-hbase4:41499] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-16 21:15:50,629 INFO [M:0;jenkins-hbase4:41499] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-16 21:15:50,630 INFO [M:0;jenkins-hbase4:41499] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-16 21:15:50,631 DEBUG [M:0;jenkins-hbase4:41499] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-16 21:15:50,631 INFO [M:0;jenkins-hbase4:41499] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:15:50,631 DEBUG [M:0;jenkins-hbase4:41499] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:15:50,631 DEBUG [M:0;jenkins-hbase4:41499] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-16 21:15:50,631 DEBUG [M:0;jenkins-hbase4:41499] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:15:50,631 INFO [M:0;jenkins-hbase4:41499] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=354.08 KB heapSize=421.62 KB 2023-07-16 21:15:50,645 INFO [M:0;jenkins-hbase4:41499] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=354.08 KB at sequenceid=772 (bloomFilter=true), to=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/4b615d5b292e4ecf9f959f15caa29d05 2023-07-16 21:15:50,651 DEBUG [M:0;jenkins-hbase4:41499] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/4b615d5b292e4ecf9f959f15caa29d05 as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/4b615d5b292e4ecf9f959f15caa29d05 2023-07-16 21:15:50,656 INFO [M:0;jenkins-hbase4:41499] regionserver.HStore(1080): Added hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/4b615d5b292e4ecf9f959f15caa29d05, entries=105, sequenceid=772, filesize=24.6 K 2023-07-16 21:15:50,657 INFO [M:0;jenkins-hbase4:41499] regionserver.HRegion(2948): Finished flush of dataSize ~354.08 KB/362575, heapSize ~421.60 KB/431720, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 26ms, sequenceid=772, compaction requested=false 2023-07-16 21:15:50,659 INFO [M:0;jenkins-hbase4:41499] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:15:50,659 DEBUG [M:0;jenkins-hbase4:41499] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-16 21:15:50,662 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:15:50,662 INFO [M:0;jenkins-hbase4:41499] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-16 21:15:50,663 INFO [M:0;jenkins-hbase4:41499] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41499 2023-07-16 21:15:50,666 DEBUG [M:0;jenkins-hbase4:41499] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,41499,1689542132027 already deleted, retry=false 2023-07-16 21:15:50,687 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:15:50,687 INFO [RS:1;jenkins-hbase4:45229] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45229,1689542134233; zookeeper connection closed. 2023-07-16 21:15:50,687 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:45229-0x1016ffd80780002, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:15:50,687 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@647cebab] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@647cebab 2023-07-16 21:15:50,787 INFO [M:0;jenkins-hbase4:41499] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41499,1689542132027; zookeeper connection closed. 2023-07-16 21:15:50,787 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:15:50,788 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:41499-0x1016ffd80780000, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:15:50,888 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:15:50,888 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36993-0x1016ffd80780001, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:15:50,888 INFO [RS:0;jenkins-hbase4:36993] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36993,1689542134044; zookeeper connection closed. 2023-07-16 21:15:50,888 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5304429b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5304429b 2023-07-16 21:15:50,888 INFO [Listener at localhost/43605] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 5 regionserver(s) complete 2023-07-16 21:15:50,888 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBasics(311): Sleeping a bit 2023-07-16 21:15:52,199 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-16 21:15:52,199 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver 2023-07-16 21:15:52,199 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint 2023-07-16 21:15:52,846 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-16 21:15:52,889 DEBUG [Listener at localhost/43605] hbase.LocalHBaseCluster(134): Setting Master Port to random. 2023-07-16 21:15:52,889 DEBUG [Listener at localhost/43605] hbase.LocalHBaseCluster(141): Setting RegionServer Port to random. 2023-07-16 21:15:52,890 DEBUG [Listener at localhost/43605] hbase.LocalHBaseCluster(151): Setting RS InfoServer Port to random. 2023-07-16 21:15:52,890 DEBUG [Listener at localhost/43605] hbase.LocalHBaseCluster(159): Setting Master InfoServer Port to random. 2023-07-16 21:15:52,891 INFO [Listener at localhost/43605] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:15:52,891 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:52,891 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:52,891 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:15:52,892 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:52,892 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:15:52,892 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:15:52,894 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33729 2023-07-16 21:15:52,895 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:52,896 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:52,898 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33729 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:15:52,902 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:337290x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:15:52,902 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:33729-0x1016ffd80780010 connected 2023-07-16 21:15:52,905 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-16 21:15:52,905 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:15:52,905 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:15:52,906 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33729 2023-07-16 21:15:52,906 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33729 2023-07-16 21:15:52,906 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33729 2023-07-16 21:15:52,909 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33729 2023-07-16 21:15:52,909 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33729 2023-07-16 21:15:52,912 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:15:52,912 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:15:52,912 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:15:52,913 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context master 2023-07-16 21:15:52,913 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:15:52,913 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:15:52,913 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:15:52,914 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 39059 2023-07-16 21:15:52,914 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:15:52,917 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:52,918 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@a8ea588{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:15:52,918 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:52,918 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@7812c246{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:15:53,035 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:15:53,037 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:15:53,037 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:15:53,037 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 600000ms 2023-07-16 21:15:53,039 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:53,040 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@34f8329c{master,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-39059-hbase-server-2_4_18-SNAPSHOT_jar-_-any-4078720027072844837/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/master} 2023-07-16 21:15:53,042 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@16af4387{HTTP/1.1, (http/1.1)}{0.0.0.0:39059} 2023-07-16 21:15:53,042 INFO [Listener at localhost/43605] server.Server(415): Started @26907ms 2023-07-16 21:15:53,042 INFO [Listener at localhost/43605] master.HMaster(444): hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb, hbase.cluster.distributed=false 2023-07-16 21:15:53,045 DEBUG [pool-340-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: INIT 2023-07-16 21:15:53,062 INFO [Listener at localhost/43605] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:15:53,062 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:53,063 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:53,063 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:15:53,063 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:53,063 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:15:53,063 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:15:53,064 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34345 2023-07-16 21:15:53,064 INFO [Listener at localhost/43605] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-16 21:15:53,065 DEBUG [Listener at localhost/43605] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-16 21:15:53,066 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:53,067 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:53,069 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34345 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:15:53,071 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:343450x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:15:53,072 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:343450x0, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-16 21:15:53,073 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34345-0x1016ffd80780011 connected 2023-07-16 21:15:53,073 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:15:53,073 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:15:53,074 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34345 2023-07-16 21:15:53,074 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34345 2023-07-16 21:15:53,074 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34345 2023-07-16 21:15:53,075 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34345 2023-07-16 21:15:53,078 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34345 2023-07-16 21:15:53,080 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:15:53,080 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:15:53,080 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:15:53,081 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-16 21:15:53,081 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:15:53,081 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:15:53,081 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:15:53,082 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 37025 2023-07-16 21:15:53,082 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:15:53,085 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:53,086 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@3e757193{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:15:53,086 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:53,086 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@2d55c5ad{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:15:53,237 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:15:53,238 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:15:53,239 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:15:53,239 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 600000ms 2023-07-16 21:15:53,240 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:53,241 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@15137874{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-37025-hbase-server-2_4_18-SNAPSHOT_jar-_-any-497026344691781155/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:15:53,242 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@33f7f8e1{HTTP/1.1, (http/1.1)}{0.0.0.0:37025} 2023-07-16 21:15:53,243 INFO [Listener at localhost/43605] server.Server(415): Started @27107ms 2023-07-16 21:15:53,257 INFO [Listener at localhost/43605] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:15:53,257 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:53,257 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:53,257 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:15:53,257 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:53,257 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:15:53,257 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:15:53,258 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41419 2023-07-16 21:15:53,258 INFO [Listener at localhost/43605] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-16 21:15:53,259 DEBUG [Listener at localhost/43605] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-16 21:15:53,260 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:53,261 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:53,262 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41419 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:15:53,266 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:414190x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:15:53,267 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:414190x0, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-16 21:15:53,268 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:414190x0, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:15:53,271 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41419-0x1016ffd80780012 connected 2023-07-16 21:15:53,271 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:41419-0x1016ffd80780012, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:15:53,275 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41419 2023-07-16 21:15:53,275 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41419 2023-07-16 21:15:53,275 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41419 2023-07-16 21:15:53,276 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41419 2023-07-16 21:15:53,278 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41419 2023-07-16 21:15:53,280 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:15:53,280 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:15:53,280 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:15:53,281 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-16 21:15:53,281 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:15:53,281 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:15:53,281 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:15:53,282 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 41479 2023-07-16 21:15:53,282 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:15:53,285 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:53,286 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@7d525552{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:15:53,286 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:53,286 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@5ec7ab45{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:15:53,406 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:15:53,407 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:15:53,407 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:15:53,407 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 600000ms 2023-07-16 21:15:53,408 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:53,409 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@4294be60{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-41479-hbase-server-2_4_18-SNAPSHOT_jar-_-any-1538952240512327595/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:15:53,411 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@79a34b80{HTTP/1.1, (http/1.1)}{0.0.0.0:41479} 2023-07-16 21:15:53,412 INFO [Listener at localhost/43605] server.Server(415): Started @27276ms 2023-07-16 21:15:53,430 INFO [Listener at localhost/43605] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:15:53,430 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:53,430 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:53,430 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:15:53,431 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:15:53,431 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:15:53,431 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:15:53,433 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34991 2023-07-16 21:15:53,433 INFO [Listener at localhost/43605] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-16 21:15:53,435 DEBUG [Listener at localhost/43605] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-16 21:15:53,435 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:53,436 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:53,437 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34991 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:15:53,444 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:349910x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:15:53,445 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:349910x0, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-16 21:15:53,446 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34991-0x1016ffd80780013 connected 2023-07-16 21:15:53,446 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:15:53,447 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:15:53,451 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34991 2023-07-16 21:15:53,452 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34991 2023-07-16 21:15:53,454 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34991 2023-07-16 21:15:53,463 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34991 2023-07-16 21:15:53,463 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34991 2023-07-16 21:15:53,466 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:15:53,466 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:15:53,466 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:15:53,467 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-16 21:15:53,467 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:15:53,467 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:15:53,467 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:15:53,468 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 39811 2023-07-16 21:15:53,468 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:15:53,471 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:53,471 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@3e81019e{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:15:53,472 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:53,472 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@7de5dfd5{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:15:53,604 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:15:53,605 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:15:53,605 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:15:53,606 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-16 21:15:53,607 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:15:53,608 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@42067a36{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-39811-hbase-server-2_4_18-SNAPSHOT_jar-_-any-5301955285759021518/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:15:53,610 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@70dbea2d{HTTP/1.1, (http/1.1)}{0.0.0.0:39811} 2023-07-16 21:15:53,610 INFO [Listener at localhost/43605] server.Server(415): Started @27474ms 2023-07-16 21:15:53,614 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:15:53,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.AbstractConnector(333): Started ServerConnector@1faa878d{HTTP/1.1, (http/1.1)}{0.0.0.0:46705} 2023-07-16 21:15:53,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.Server(415): Started @27492ms 2023-07-16 21:15:53,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,33729,1689542152890 2023-07-16 21:15:53,629 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-16 21:15:53,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,33729,1689542152890 2023-07-16 21:15:53,631 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-16 21:15:53,631 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41419-0x1016ffd80780012, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-16 21:15:53,631 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-16 21:15:53,631 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-16 21:15:53,632 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:15:53,634 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-16 21:15:53,635 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-16 21:15:53,635 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,33729,1689542152890 from backup master directory 2023-07-16 21:15:53,636 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,33729,1689542152890 2023-07-16 21:15:53,636 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-16 21:15:53,636 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:15:53,636 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,33729,1689542152890 2023-07-16 21:15:53,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:15:53,694 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x5c6ed03f to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:15:53,707 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@a2c1f9e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:15:53,707 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-16 21:15:53,708 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-16 21:15:53,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:15:53,720 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,41499,1689542132027 to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,41499,1689542132027-dead as it is dead 2023-07-16 21:15:53,723 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,41499,1689542132027-dead/jenkins-hbase4.apache.org%2C41499%2C1689542132027.1689542135245 2023-07-16 21:15:53,729 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,41499,1689542132027-dead/jenkins-hbase4.apache.org%2C41499%2C1689542132027.1689542135245 after 6ms 2023-07-16 21:15:53,731 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,41499,1689542132027-dead/jenkins-hbase4.apache.org%2C41499%2C1689542132027.1689542135245 to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C41499%2C1689542132027.1689542135245 2023-07-16 21:15:53,731 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,41499,1689542132027-dead 2023-07-16 21:15:53,732 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,33729,1689542152890 2023-07-16 21:15:53,735 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33729%2C1689542152890, suffix=, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,33729,1689542152890, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/oldWALs, maxLogs=10 2023-07-16 21:15:53,753 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:15:53,755 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:15:53,754 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:15:53,767 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,33729,1689542152890/jenkins-hbase4.apache.org%2C33729%2C1689542152890.1689542153735 2023-07-16 21:15:53,770 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK], DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK], DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK]] 2023-07-16 21:15:53,771 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:15:53,771 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:53,771 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-16 21:15:53,771 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-16 21:15:53,776 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-16 21:15:53,777 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-16 21:15:53,778 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-16 21:15:53,788 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/4b615d5b292e4ecf9f959f15caa29d05 2023-07-16 21:15:53,789 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:53,790 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-16 21:15:53,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C41499%2C1689542132027.1689542135245 2023-07-16 21:15:53,829 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 911, firstSequenceIdInLog=3, maxSequenceIdInLog=774, path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C41499%2C1689542132027.1689542135245 2023-07-16 21:15:53,831 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C41499%2C1689542132027.1689542135245 2023-07-16 21:15:53,843 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-16 21:15:53,847 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/774.seqid, newMaxSeqId=774, maxSeqId=1 2023-07-16 21:15:53,848 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=775; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11449950400, jitterRate=0.06635972857475281}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:53,849 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-16 21:15:53,851 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-16 21:15:53,852 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-16 21:15:53,852 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-16 21:15:53,852 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-16 21:15:53,853 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-16 21:15:53,877 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-16 21:15:53,878 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-16 21:15:53,878 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=6, state=SUCCESS; CreateTableProcedure table=hbase:rsgroup 2023-07-16 21:15:53,878 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=10, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-16 21:15:53,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=11, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-16 21:15:53,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,33295,1689542138167, splitWal=true, meta=false 2023-07-16 21:15:53,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=13, state=SUCCESS; ModifyNamespaceProcedure, namespace=default 2023-07-16 21:15:53,880 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=14, state=SUCCESS; CreateTableProcedure table=Group_testCreateAndAssign 2023-07-16 21:15:53,880 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=17, state=SUCCESS; DisableTableProcedure table=Group_testCreateAndAssign 2023-07-16 21:15:53,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=20, state=SUCCESS; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-16 21:15:53,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=21, state=SUCCESS; CreateTableProcedure table=Group_testCreateMultiRegion 2023-07-16 21:15:53,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=42, state=SUCCESS; DisableTableProcedure table=Group_testCreateMultiRegion 2023-07-16 21:15:53,882 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-16 21:15:53,882 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=64, state=SUCCESS; CreateNamespaceProcedure, namespace=Group_foo 2023-07-16 21:15:53,882 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=65, state=SUCCESS; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-16 21:15:53,882 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DisableTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-16 21:15:53,883 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=71, state=SUCCESS; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-16 21:15:53,883 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-16 21:15:53,883 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=73, state=SUCCESS; CreateTableProcedure table=Group_testCreateAndDrop 2023-07-16 21:15:53,884 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=76, state=SUCCESS; DisableTableProcedure table=Group_testCreateAndDrop 2023-07-16 21:15:53,884 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-16 21:15:53,884 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=80, state=SUCCESS; CreateTableProcedure table=Group_testCloneSnapshot 2023-07-16 21:15:53,884 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=83, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=EXCLUSIVE 2023-07-16 21:15:53,885 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=SHARED 2023-07-16 21:15:53,885 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CloneSnapshotProcedure (table=Group_testCloneSnapshot_clone snapshot=name: "Group_testCloneSnapshot_snap" table: "Group_testCloneSnapshot" creation_time: 1689542145702 type: FLUSH version: 2 ttl: 0 ) 2023-07-16 21:15:53,885 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; DisableTableProcedure table=Group_testCloneSnapshot 2023-07-16 21:15:53,886 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=91, state=SUCCESS; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-16 21:15:53,886 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=92, state=SUCCESS; DisableTableProcedure table=Group_testCloneSnapshot_clone 2023-07-16 21:15:53,886 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=95, state=SUCCESS; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-16 21:15:53,886 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; CreateNamespaceProcedure, namespace=Group_ns 2023-07-16 21:15:53,887 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=97, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-create-table:org.apache.hadoop.hbase.HBaseIOException: No online servers in the rsgroup appInfo which table Group_ns:testCreateWhenRsgroupNoOnlineServers belongs to; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:53,887 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=98, state=SUCCESS; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:53,887 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DisableTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:53,887 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=104, state=SUCCESS; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:15:53,888 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-16 21:15:53,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 34 msec 2023-07-16 21:15:53,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-16 21:15:53,889 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-16 21:15:53,889 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36993,1689542134044, table=hbase:meta, region=1588230740 2023-07-16 21:15:53,891 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 4 possibly 'live' servers, and 0 'splitting'. 2023-07-16 21:15:53,892 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41213,1689542138767 already deleted, retry=false 2023-07-16 21:15:53,892 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,41213,1689542138767 on jenkins-hbase4.apache.org,33729,1689542152890 2023-07-16 21:15:53,893 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=106, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,41213,1689542138767, splitWal=true, meta=false 2023-07-16 21:15:53,893 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=106 for jenkins-hbase4.apache.org,41213,1689542138767 (carryingMeta=false) jenkins-hbase4.apache.org,41213,1689542138767/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@50dc3ca4[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-16 21:15:53,895 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,45229,1689542134233 already deleted, retry=false 2023-07-16 21:15:53,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,45229,1689542134233 on jenkins-hbase4.apache.org,33729,1689542152890 2023-07-16 21:15:53,900 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=107, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,45229,1689542134233, splitWal=true, meta=false 2023-07-16 21:15:53,900 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=107 for jenkins-hbase4.apache.org,45229,1689542134233 (carryingMeta=false) jenkins-hbase4.apache.org,45229,1689542134233/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@157c65d4[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-16 21:15:53,901 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,33511,1689542134438 already deleted, retry=false 2023-07-16 21:15:53,901 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,33511,1689542134438 on jenkins-hbase4.apache.org,33729,1689542152890 2023-07-16 21:15:53,902 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,33511,1689542134438, splitWal=true, meta=false 2023-07-16 21:15:53,902 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=108 for jenkins-hbase4.apache.org,33511,1689542134438 (carryingMeta=false) jenkins-hbase4.apache.org,33511,1689542134438/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@39e495de[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-16 21:15:53,903 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36993,1689542134044 already deleted, retry=false 2023-07-16 21:15:53,903 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,36993,1689542134044 on jenkins-hbase4.apache.org,33729,1689542152890 2023-07-16 21:15:53,904 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,36993,1689542134044, splitWal=true, meta=true 2023-07-16 21:15:53,904 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=109 for jenkins-hbase4.apache.org,36993,1689542134044 (carryingMeta=true) jenkins-hbase4.apache.org,36993,1689542134044/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@379b5fcc[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-16 21:15:53,905 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/balancer 2023-07-16 21:15:53,906 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-16 21:15:53,906 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-07-16 21:15:53,907 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-07-16 21:15:53,907 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-07-16 21:15:53,908 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-07-16 21:15:53,911 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-16 21:15:53,911 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41419-0x1016ffd80780012, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-16 21:15:53,911 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-16 21:15:53,911 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-16 21:15:53,911 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:15:53,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,33729,1689542152890, sessionid=0x1016ffd80780010, setting cluster-up flag (Was=false) 2023-07-16 21:15:53,916 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-07-16 21:15:53,917 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33729,1689542152890 2023-07-16 21:15:53,920 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-07-16 21:15:53,920 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33729,1689542152890 2023-07-16 21:15:53,924 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2930): Registered master coprocessor service: service=RSGroupAdminService 2023-07-16 21:15:53,924 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] rsgroup.RSGroupInfoManagerImpl(537): Refreshing in Offline mode. 2023-07-16 21:15:53,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] rsgroup.RSGroupInfoManagerImpl(511): Read ZK GroupInfo count:2 2023-07-16 21:15:53,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint loaded, priority=536870911. 2023-07-16 21:15:53,926 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33729,1689542152890] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:15:53,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver loaded, priority=536870912. 2023-07-16 21:15:53,927 INFO [master/jenkins-hbase4:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.quotas.MasterQuotasObserver loaded, priority=536870913. 2023-07-16 21:15:53,943 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33729,1689542152890] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:15:53,944 WARN [RS-EventLoopGroup-12-3] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase4.apache.org/172.31.14.131:36993 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:36993 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-16 21:15:53,947 DEBUG [RS-EventLoopGroup-12-3] ipc.FailedServers(52): Added failed server with address jenkins-hbase4.apache.org/172.31.14.131:36993 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:36993 2023-07-16 21:15:53,948 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-16 21:15:53,948 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-16 21:15:53,948 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-16 21:15:53,948 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-16 21:15:53,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-16 21:15:53,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-16 21:15:53,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-16 21:15:53,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-16 21:15:53,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-16 21:15:53,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:53,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:15:53,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:53,958 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689542183958 2023-07-16 21:15:53,963 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36993,1689542134044; numProcessing=1 2023-07-16 21:15:53,964 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=109, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,36993,1689542134044, splitWal=true, meta=true 2023-07-16 21:15:53,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-16 21:15:53,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-16 21:15:53,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-16 21:15:53,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-16 21:15:53,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-16 21:15:53,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-16 21:15:53,965 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:53,965 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45229,1689542134233; numProcessing=2 2023-07-16 21:15:53,966 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=107, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45229,1689542134233, splitWal=true, meta=false 2023-07-16 21:15:53,966 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-16 21:15:53,966 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41213,1689542138767; numProcessing=3 2023-07-16 21:15:53,967 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=106, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,41213,1689542138767, splitWal=true, meta=false 2023-07-16 21:15:53,966 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-16 21:15:53,967 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=109, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,36993,1689542134044, splitWal=true, meta=true, isMeta: true 2023-07-16 21:15:53,972 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-16 21:15:53,967 DEBUG [PEWorker-4] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33511,1689542134438; numProcessing=4 2023-07-16 21:15:53,973 INFO [PEWorker-4] procedure.ServerCrashProcedure(161): Start pid=108, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,33511,1689542134438, splitWal=true, meta=false 2023-07-16 21:15:53,974 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-16 21:15:53,974 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-16 21:15:53,975 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,36993,1689542134044-splitting 2023-07-16 21:15:53,976 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,36993,1689542134044-splitting dir is empty, no logs to split. 2023-07-16 21:15:53,977 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,36993,1689542134044 WAL count=0, meta=true 2023-07-16 21:15:53,980 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,36993,1689542134044-splitting dir is empty, no logs to split. 2023-07-16 21:15:53,981 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,36993,1689542134044 WAL count=0, meta=true 2023-07-16 21:15:53,981 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,36993,1689542134044 WAL splitting is done? wals=0, meta=true 2023-07-16 21:15:53,982 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689542153974,5,FailOnTimeoutGroup] 2023-07-16 21:15:53,984 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-16 21:15:53,986 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689542153983,5,FailOnTimeoutGroup] 2023-07-16 21:15:53,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:53,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-16 21:15:53,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:53,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:53,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689542153987, completionTime=-1 2023-07-16 21:15:53,987 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-16 21:15:53,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-16 21:15:53,989 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-16 21:15:53,990 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-16 21:15:54,016 INFO [RS:1;jenkins-hbase4:41419] regionserver.HRegionServer(951): ClusterId : bc8d5a8a-d305-489f-bc1d-b95b98296a6c 2023-07-16 21:15:54,016 INFO [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(951): ClusterId : bc8d5a8a-d305-489f-bc1d-b95b98296a6c 2023-07-16 21:15:54,016 INFO [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(951): ClusterId : bc8d5a8a-d305-489f-bc1d-b95b98296a6c 2023-07-16 21:15:54,018 DEBUG [RS:1;jenkins-hbase4:41419] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-16 21:15:54,019 DEBUG [RS:0;jenkins-hbase4:34345] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-16 21:15:54,019 DEBUG [RS:2;jenkins-hbase4:34991] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-16 21:15:54,023 DEBUG [RS:1;jenkins-hbase4:41419] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-16 21:15:54,023 DEBUG [RS:1;jenkins-hbase4:41419] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-16 21:15:54,027 DEBUG [RS:0;jenkins-hbase4:34345] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-16 21:15:54,027 DEBUG [RS:0;jenkins-hbase4:34345] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-16 21:15:54,027 DEBUG [RS:2;jenkins-hbase4:34991] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-16 21:15:54,027 DEBUG [RS:2;jenkins-hbase4:34991] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-16 21:15:54,030 DEBUG [RS:0;jenkins-hbase4:34345] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-16 21:15:54,031 DEBUG [RS:2;jenkins-hbase4:34991] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-16 21:15:54,031 DEBUG [RS:1;jenkins-hbase4:41419] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-16 21:15:54,031 DEBUG [RS:0;jenkins-hbase4:34345] zookeeper.ReadOnlyZKClient(139): Connect 0x2a68c75d to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:15:54,033 DEBUG [RS:1;jenkins-hbase4:41419] zookeeper.ReadOnlyZKClient(139): Connect 0x587ac046 to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:15:54,033 DEBUG [RS:2;jenkins-hbase4:34991] zookeeper.ReadOnlyZKClient(139): Connect 0x13afa02a to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:15:54,049 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33729,1689542152890] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:36993 this server is in the failed servers list 2023-07-16 21:15:54,061 DEBUG [RS:1;jenkins-hbase4:41419] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@56bd3539, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:15:54,061 DEBUG [RS:1;jenkins-hbase4:41419] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5282ab36, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:15:54,061 DEBUG [RS:2;jenkins-hbase4:34991] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d685c2f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:15:54,062 DEBUG [RS:2;jenkins-hbase4:34991] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1957cb5a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:15:54,070 DEBUG [RS:0;jenkins-hbase4:34345] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@24af2c60, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:15:54,071 DEBUG [RS:0;jenkins-hbase4:34345] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2090ac2d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:15:54,073 DEBUG [RS:1;jenkins-hbase4:41419] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:41419 2023-07-16 21:15:54,074 INFO [RS:1;jenkins-hbase4:41419] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-16 21:15:54,074 INFO [RS:1;jenkins-hbase4:41419] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-16 21:15:54,074 DEBUG [RS:1;jenkins-hbase4:41419] regionserver.HRegionServer(1022): About to register with Master. 2023-07-16 21:15:54,074 DEBUG [RS:2;jenkins-hbase4:34991] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:34991 2023-07-16 21:15:54,075 INFO [RS:2;jenkins-hbase4:34991] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-16 21:15:54,075 INFO [RS:2;jenkins-hbase4:34991] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-16 21:15:54,075 DEBUG [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(1022): About to register with Master. 2023-07-16 21:15:54,075 INFO [RS:1;jenkins-hbase4:41419] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33729,1689542152890 with isa=jenkins-hbase4.apache.org/172.31.14.131:41419, startcode=1689542153256 2023-07-16 21:15:54,075 DEBUG [RS:1;jenkins-hbase4:41419] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-16 21:15:54,075 INFO [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33729,1689542152890 with isa=jenkins-hbase4.apache.org/172.31.14.131:34991, startcode=1689542153429 2023-07-16 21:15:54,076 DEBUG [RS:2;jenkins-hbase4:34991] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-16 21:15:54,079 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58017, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-07-16 21:15:54,083 DEBUG [RS:0;jenkins-hbase4:34345] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:34345 2023-07-16 21:15:54,083 INFO [RS:0;jenkins-hbase4:34345] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-16 21:15:54,083 INFO [RS:0;jenkins-hbase4:34345] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-16 21:15:54,083 DEBUG [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(1022): About to register with Master. 2023-07-16 21:15:54,083 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50549, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.7 (auth:SIMPLE), service=RegionServerStatusService 2023-07-16 21:15:54,084 INFO [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33729,1689542152890 with isa=jenkins-hbase4.apache.org/172.31.14.131:34345, startcode=1689542153062 2023-07-16 21:15:54,086 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33729] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41419,1689542153256 2023-07-16 21:15:54,086 DEBUG [RS:0;jenkins-hbase4:34345] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-16 21:15:54,086 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33729,1689542152890] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:15:54,087 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33729,1689542152890] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 1 2023-07-16 21:15:54,087 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33729] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:15:54,087 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33729,1689542152890] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:15:54,087 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33729,1689542152890] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 2 2023-07-16 21:15:54,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=2; waited=101ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-16 21:15:54,088 DEBUG [RS:1;jenkins-hbase4:41419] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb 2023-07-16 21:15:54,088 DEBUG [RS:1;jenkins-hbase4:41419] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35261 2023-07-16 21:15:54,088 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56979, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-07-16 21:15:54,088 DEBUG [RS:1;jenkins-hbase4:41419] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=39059 2023-07-16 21:15:54,088 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33729] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34345,1689542153062 2023-07-16 21:15:54,088 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33729,1689542152890] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:15:54,088 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33729,1689542152890] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 3 2023-07-16 21:15:54,090 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:15:54,090 DEBUG [RS:1;jenkins-hbase4:41419] zookeeper.ZKUtil(162): regionserver:41419-0x1016ffd80780012, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41419,1689542153256 2023-07-16 21:15:54,090 WARN [RS:1;jenkins-hbase4:41419] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:15:54,090 INFO [RS:1;jenkins-hbase4:41419] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:15:54,091 DEBUG [RS:1;jenkins-hbase4:41419] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,41419,1689542153256 2023-07-16 21:15:54,091 DEBUG [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb 2023-07-16 21:15:54,091 DEBUG [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35261 2023-07-16 21:15:54,091 DEBUG [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=39059 2023-07-16 21:15:54,091 DEBUG [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb 2023-07-16 21:15:54,091 DEBUG [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35261 2023-07-16 21:15:54,091 DEBUG [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=39059 2023-07-16 21:15:54,096 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34345,1689542153062] 2023-07-16 21:15:54,096 DEBUG [RS:0;jenkins-hbase4:34345] zookeeper.ZKUtil(162): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34345,1689542153062 2023-07-16 21:15:54,096 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41419,1689542153256] 2023-07-16 21:15:54,096 DEBUG [RS:2;jenkins-hbase4:34991] zookeeper.ZKUtil(162): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:15:54,096 WARN [RS:0;jenkins-hbase4:34345] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:15:54,096 WARN [RS:2;jenkins-hbase4:34991] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:15:54,096 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34991,1689542153429] 2023-07-16 21:15:54,096 INFO [RS:2;jenkins-hbase4:34991] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:15:54,096 INFO [RS:0;jenkins-hbase4:34345] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:15:54,096 DEBUG [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:15:54,096 DEBUG [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34345,1689542153062 2023-07-16 21:15:54,106 DEBUG [RS:1;jenkins-hbase4:41419] zookeeper.ZKUtil(162): regionserver:41419-0x1016ffd80780012, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34345,1689542153062 2023-07-16 21:15:54,107 DEBUG [RS:0;jenkins-hbase4:34345] zookeeper.ZKUtil(162): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34345,1689542153062 2023-07-16 21:15:54,107 DEBUG [RS:2;jenkins-hbase4:34991] zookeeper.ZKUtil(162): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34345,1689542153062 2023-07-16 21:15:54,107 DEBUG [RS:1;jenkins-hbase4:41419] zookeeper.ZKUtil(162): regionserver:41419-0x1016ffd80780012, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41419,1689542153256 2023-07-16 21:15:54,107 DEBUG [RS:0;jenkins-hbase4:34345] zookeeper.ZKUtil(162): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41419,1689542153256 2023-07-16 21:15:54,107 DEBUG [RS:2;jenkins-hbase4:34991] zookeeper.ZKUtil(162): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41419,1689542153256 2023-07-16 21:15:54,107 DEBUG [RS:1;jenkins-hbase4:41419] zookeeper.ZKUtil(162): regionserver:41419-0x1016ffd80780012, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:15:54,107 DEBUG [RS:0;jenkins-hbase4:34345] zookeeper.ZKUtil(162): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:15:54,108 DEBUG [RS:2;jenkins-hbase4:34991] zookeeper.ZKUtil(162): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:15:54,108 DEBUG [RS:1;jenkins-hbase4:41419] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-16 21:15:54,109 INFO [RS:1;jenkins-hbase4:41419] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-16 21:15:54,109 DEBUG [RS:0;jenkins-hbase4:34345] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-16 21:15:54,109 DEBUG [RS:2;jenkins-hbase4:34991] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-16 21:15:54,110 INFO [RS:2;jenkins-hbase4:34991] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-16 21:15:54,111 INFO [RS:0;jenkins-hbase4:34345] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-16 21:15:54,115 INFO [RS:1;jenkins-hbase4:41419] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-16 21:15:54,115 INFO [RS:0;jenkins-hbase4:34345] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-16 21:15:54,119 INFO [RS:2;jenkins-hbase4:34991] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-16 21:15:54,120 INFO [RS:1;jenkins-hbase4:41419] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-16 21:15:54,120 INFO [RS:0;jenkins-hbase4:34345] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-16 21:15:54,120 INFO [RS:1;jenkins-hbase4:41419] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,120 INFO [RS:0;jenkins-hbase4:34345] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,120 INFO [RS:2;jenkins-hbase4:34991] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-16 21:15:54,121 INFO [RS:2;jenkins-hbase4:34991] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,121 INFO [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-16 21:15:54,122 INFO [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-16 21:15:54,127 INFO [RS:1;jenkins-hbase4:41419] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-16 21:15:54,129 INFO [RS:0;jenkins-hbase4:34345] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,130 INFO [RS:1;jenkins-hbase4:41419] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,130 INFO [RS:2;jenkins-hbase4:34991] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,130 DEBUG [RS:1;jenkins-hbase4:41419] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,130 DEBUG [RS:2;jenkins-hbase4:34991] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,130 DEBUG [RS:2;jenkins-hbase4:34991] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,130 DEBUG [RS:2;jenkins-hbase4:34991] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,130 DEBUG [RS:2;jenkins-hbase4:34991] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,130 DEBUG [RS:2;jenkins-hbase4:34991] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,130 DEBUG [RS:2;jenkins-hbase4:34991] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:15:54,130 DEBUG [RS:2;jenkins-hbase4:34991] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,130 DEBUG [RS:0;jenkins-hbase4:34345] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,130 DEBUG [RS:2;jenkins-hbase4:34991] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,131 DEBUG [RS:0;jenkins-hbase4:34345] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,131 DEBUG [RS:2;jenkins-hbase4:34991] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,130 DEBUG [RS:1;jenkins-hbase4:41419] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,131 DEBUG [RS:2;jenkins-hbase4:34991] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,131 DEBUG [RS:1;jenkins-hbase4:41419] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,131 DEBUG [RS:1;jenkins-hbase4:41419] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,131 DEBUG [RS:1;jenkins-hbase4:41419] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,131 DEBUG [RS:1;jenkins-hbase4:41419] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:15:54,131 DEBUG [RS:1;jenkins-hbase4:41419] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,131 DEBUG [RS:1;jenkins-hbase4:41419] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,131 DEBUG [RS:1;jenkins-hbase4:41419] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,132 DEBUG [RS:1;jenkins-hbase4:41419] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,131 DEBUG [RS:0;jenkins-hbase4:34345] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,132 DEBUG [RS:0;jenkins-hbase4:34345] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,132 DEBUG [RS:0;jenkins-hbase4:34345] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,132 DEBUG [RS:0;jenkins-hbase4:34345] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:15:54,132 DEBUG [RS:0;jenkins-hbase4:34345] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,132 DEBUG [RS:0;jenkins-hbase4:34345] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,132 DEBUG [RS:0;jenkins-hbase4:34345] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,132 DEBUG [RS:0;jenkins-hbase4:34345] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:15:54,138 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=3; waited=151ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-16 21:15:54,140 INFO [RS:1;jenkins-hbase4:41419] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,140 INFO [RS:1;jenkins-hbase4:41419] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,140 INFO [RS:1;jenkins-hbase4:41419] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,140 INFO [RS:1;jenkins-hbase4:41419] hbase.ChoreService(166): Chore ScheduledChore name=FileSystemUtilizationChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,140 DEBUG [jenkins-hbase4:33729] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-07-16 21:15:54,140 INFO [RS:2;jenkins-hbase4:34991] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,140 DEBUG [jenkins-hbase4:33729] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-16 21:15:54,141 INFO [RS:2;jenkins-hbase4:34991] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,141 DEBUG [jenkins-hbase4:33729] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-16 21:15:54,141 INFO [RS:2;jenkins-hbase4:34991] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,141 INFO [RS:2;jenkins-hbase4:34991] hbase.ChoreService(166): Chore ScheduledChore name=FileSystemUtilizationChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,141 DEBUG [jenkins-hbase4:33729] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-16 21:15:54,141 DEBUG [jenkins-hbase4:33729] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-16 21:15:54,141 DEBUG [jenkins-hbase4:33729] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-16 21:15:54,142 INFO [RS:0;jenkins-hbase4:34345] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,142 INFO [RS:0;jenkins-hbase4:34345] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,142 INFO [RS:0;jenkins-hbase4:34345] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,142 INFO [RS:0;jenkins-hbase4:34345] hbase.ChoreService(166): Chore ScheduledChore name=FileSystemUtilizationChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,159 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34991,1689542153429, state=OPENING 2023-07-16 21:15:54,161 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-16 21:15:54,161 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-16 21:15:54,163 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34991,1689542153429}] 2023-07-16 21:15:54,174 INFO [RS:1;jenkins-hbase4:41419] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-16 21:15:54,174 INFO [RS:1;jenkins-hbase4:41419] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41419,1689542153256-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,177 INFO [RS:0;jenkins-hbase4:34345] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-16 21:15:54,177 INFO [RS:0;jenkins-hbase4:34345] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34345,1689542153062-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,185 INFO [RS:2;jenkins-hbase4:34991] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-16 21:15:54,185 INFO [RS:2;jenkins-hbase4:34991] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34991,1689542153429-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,187 INFO [RS:1;jenkins-hbase4:41419] regionserver.Replication(203): jenkins-hbase4.apache.org,41419,1689542153256 started 2023-07-16 21:15:54,187 INFO [RS:1;jenkins-hbase4:41419] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,41419,1689542153256, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41419, sessionid=0x1016ffd80780012 2023-07-16 21:15:54,187 DEBUG [RS:1;jenkins-hbase4:41419] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-16 21:15:54,187 DEBUG [RS:1;jenkins-hbase4:41419] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41419,1689542153256 2023-07-16 21:15:54,187 DEBUG [RS:1;jenkins-hbase4:41419] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41419,1689542153256' 2023-07-16 21:15:54,187 DEBUG [RS:1;jenkins-hbase4:41419] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-16 21:15:54,187 DEBUG [RS:1;jenkins-hbase4:41419] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-16 21:15:54,188 DEBUG [RS:1;jenkins-hbase4:41419] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-16 21:15:54,188 DEBUG [RS:1;jenkins-hbase4:41419] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-16 21:15:54,188 DEBUG [RS:1;jenkins-hbase4:41419] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41419,1689542153256 2023-07-16 21:15:54,188 DEBUG [RS:1;jenkins-hbase4:41419] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41419,1689542153256' 2023-07-16 21:15:54,188 DEBUG [RS:1;jenkins-hbase4:41419] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:54,188 INFO [RS:0;jenkins-hbase4:34345] regionserver.Replication(203): jenkins-hbase4.apache.org,34345,1689542153062 started 2023-07-16 21:15:54,188 DEBUG [RS:1;jenkins-hbase4:41419] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:54,188 INFO [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34345,1689542153062, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34345, sessionid=0x1016ffd80780011 2023-07-16 21:15:54,188 DEBUG [RS:0;jenkins-hbase4:34345] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-16 21:15:54,188 DEBUG [RS:0;jenkins-hbase4:34345] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34345,1689542153062 2023-07-16 21:15:54,188 DEBUG [RS:0;jenkins-hbase4:34345] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34345,1689542153062' 2023-07-16 21:15:54,188 DEBUG [RS:0;jenkins-hbase4:34345] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-16 21:15:54,188 DEBUG [RS:1;jenkins-hbase4:41419] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-16 21:15:54,189 INFO [RS:1;jenkins-hbase4:41419] quotas.RegionServerRpcQuotaManager(67): Initializing RPC quota support 2023-07-16 21:15:54,189 DEBUG [RS:0;jenkins-hbase4:34345] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-16 21:15:54,189 DEBUG [RS:0;jenkins-hbase4:34345] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-16 21:15:54,189 DEBUG [RS:0;jenkins-hbase4:34345] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-16 21:15:54,189 DEBUG [RS:0;jenkins-hbase4:34345] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34345,1689542153062 2023-07-16 21:15:54,189 DEBUG [RS:0;jenkins-hbase4:34345] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34345,1689542153062' 2023-07-16 21:15:54,189 DEBUG [RS:0;jenkins-hbase4:34345] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:54,189 DEBUG [RS:0;jenkins-hbase4:34345] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:54,190 DEBUG [RS:0;jenkins-hbase4:34345] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-16 21:15:54,190 INFO [RS:0;jenkins-hbase4:34345] quotas.RegionServerRpcQuotaManager(67): Initializing RPC quota support 2023-07-16 21:15:54,191 INFO [RS:0;jenkins-hbase4:34345] hbase.ChoreService(166): Chore ScheduledChore name=QuotaRefresherChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,191 INFO [RS:1;jenkins-hbase4:41419] hbase.ChoreService(166): Chore ScheduledChore name=QuotaRefresherChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,191 DEBUG [RS:0;jenkins-hbase4:34345] zookeeper.ZKUtil(398): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Unable to get data of znode /hbase/rpc-throttle because node does not exist (not an error) 2023-07-16 21:15:54,191 DEBUG [RS:1;jenkins-hbase4:41419] zookeeper.ZKUtil(398): regionserver:41419-0x1016ffd80780012, quorum=127.0.0.1:63917, baseZNode=/hbase Unable to get data of znode /hbase/rpc-throttle because node does not exist (not an error) 2023-07-16 21:15:54,191 INFO [RS:0;jenkins-hbase4:34345] quotas.RegionServerRpcQuotaManager(73): Start rpc quota manager and rpc throttle enabled is true 2023-07-16 21:15:54,191 INFO [RS:1;jenkins-hbase4:41419] quotas.RegionServerRpcQuotaManager(73): Start rpc quota manager and rpc throttle enabled is true 2023-07-16 21:15:54,192 INFO [RS:0;jenkins-hbase4:34345] hbase.ChoreService(166): Chore ScheduledChore name=SpaceQuotaRefresherChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,192 INFO [RS:1;jenkins-hbase4:41419] hbase.ChoreService(166): Chore ScheduledChore name=SpaceQuotaRefresherChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,192 INFO [RS:1;jenkins-hbase4:41419] hbase.ChoreService(166): Chore ScheduledChore name=RegionSizeReportingChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,192 INFO [RS:0;jenkins-hbase4:34345] hbase.ChoreService(166): Chore ScheduledChore name=RegionSizeReportingChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,202 INFO [RS:2;jenkins-hbase4:34991] regionserver.Replication(203): jenkins-hbase4.apache.org,34991,1689542153429 started 2023-07-16 21:15:54,202 INFO [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34991,1689542153429, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34991, sessionid=0x1016ffd80780013 2023-07-16 21:15:54,202 DEBUG [RS:2;jenkins-hbase4:34991] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-16 21:15:54,202 DEBUG [RS:2;jenkins-hbase4:34991] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:15:54,202 DEBUG [RS:2;jenkins-hbase4:34991] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34991,1689542153429' 2023-07-16 21:15:54,202 DEBUG [RS:2;jenkins-hbase4:34991] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-16 21:15:54,203 DEBUG [RS:2;jenkins-hbase4:34991] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-16 21:15:54,203 DEBUG [RS:2;jenkins-hbase4:34991] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-16 21:15:54,203 DEBUG [RS:2;jenkins-hbase4:34991] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-16 21:15:54,203 DEBUG [RS:2;jenkins-hbase4:34991] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:15:54,203 DEBUG [RS:2;jenkins-hbase4:34991] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34991,1689542153429' 2023-07-16 21:15:54,203 DEBUG [RS:2;jenkins-hbase4:34991] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:15:54,204 DEBUG [RS:2;jenkins-hbase4:34991] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:15:54,204 DEBUG [RS:2;jenkins-hbase4:34991] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-16 21:15:54,204 INFO [RS:2;jenkins-hbase4:34991] quotas.RegionServerRpcQuotaManager(67): Initializing RPC quota support 2023-07-16 21:15:54,204 INFO [RS:2;jenkins-hbase4:34991] hbase.ChoreService(166): Chore ScheduledChore name=QuotaRefresherChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,204 DEBUG [RS:2;jenkins-hbase4:34991] zookeeper.ZKUtil(398): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Unable to get data of znode /hbase/rpc-throttle because node does not exist (not an error) 2023-07-16 21:15:54,204 INFO [RS:2;jenkins-hbase4:34991] quotas.RegionServerRpcQuotaManager(73): Start rpc quota manager and rpc throttle enabled is true 2023-07-16 21:15:54,204 INFO [RS:2;jenkins-hbase4:34991] hbase.ChoreService(166): Chore ScheduledChore name=SpaceQuotaRefresherChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,204 INFO [RS:2;jenkins-hbase4:34991] hbase.ChoreService(166): Chore ScheduledChore name=RegionSizeReportingChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:54,250 WARN [ReadOnlyZKClient-127.0.0.1:63917@0x5c6ed03f] client.ZKConnectionRegistry(168): Meta region is in state OPENING 2023-07-16 21:15:54,251 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33729,1689542152890] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:15:54,252 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57372, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:15:54,252 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34991] ipc.CallRunner(144): callId: 2 service: ClientService methodName: Get size: 88 connection: 172.31.14.131:57372 deadline: 1689542214252, exception=org.apache.hadoop.hbase.NotServingRegionException: hbase:meta,,1 is not online on jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:15:54,295 INFO [RS:1;jenkins-hbase4:41419] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41419%2C1689542153256, suffix=, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,41419,1689542153256, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs, maxLogs=32 2023-07-16 21:15:54,295 INFO [RS:0;jenkins-hbase4:34345] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34345%2C1689542153062, suffix=, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34345,1689542153062, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs, maxLogs=32 2023-07-16 21:15:54,306 INFO [RS:2;jenkins-hbase4:34991] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34991%2C1689542153429, suffix=, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34991,1689542153429, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs, maxLogs=32 2023-07-16 21:15:54,319 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:15:54,319 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:15:54,319 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:15:54,322 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:15:54,323 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-16 21:15:54,325 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57384, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-16 21:15:54,326 INFO [RS:1;jenkins-hbase4:41419] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,41419,1689542153256/jenkins-hbase4.apache.org%2C41419%2C1689542153256.1689542154301 2023-07-16 21:15:54,328 DEBUG [RS:1;jenkins-hbase4:41419] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK], DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK], DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK]] 2023-07-16 21:15:54,338 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-16 21:15:54,338 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:15:54,338 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:15:54,338 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:15:54,338 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:15:54,339 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:15:54,339 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:15:54,339 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:15:54,341 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34991%2C1689542153429.meta, suffix=.meta, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34991,1689542153429, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs, maxLogs=32 2023-07-16 21:15:54,347 INFO [RS:0;jenkins-hbase4:34345] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34345,1689542153062/jenkins-hbase4.apache.org%2C34345%2C1689542153062.1689542154301 2023-07-16 21:15:54,353 DEBUG [RS:0;jenkins-hbase4:34345] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK], DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK], DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK]] 2023-07-16 21:15:54,353 INFO [RS:2;jenkins-hbase4:34991] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34991,1689542153429/jenkins-hbase4.apache.org%2C34991%2C1689542153429.1689542154307 2023-07-16 21:15:54,354 DEBUG [RS:2;jenkins-hbase4:34991] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK], DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK], DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK]] 2023-07-16 21:15:54,357 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:15:54,357 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:15:54,357 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:15:54,363 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34991,1689542153429/jenkins-hbase4.apache.org%2C34991%2C1689542153429.meta.1689542154341.meta 2023-07-16 21:15:54,364 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK], DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK], DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK]] 2023-07-16 21:15:54,364 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:15:54,364 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-16 21:15:54,365 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-16 21:15:54,365 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-16 21:15:54,365 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-16 21:15:54,365 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:54,365 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-16 21:15:54,365 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-16 21:15:54,371 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-16 21:15:54,372 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info 2023-07-16 21:15:54,372 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info 2023-07-16 21:15:54,372 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-16 21:15:54,382 INFO [StoreFileOpener-info-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c49957a900a44abbf01b64d8b340ddd 2023-07-16 21:15:54,382 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/8c49957a900a44abbf01b64d8b340ddd 2023-07-16 21:15:54,382 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:54,382 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-16 21:15:54,383 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/rep_barrier 2023-07-16 21:15:54,384 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/rep_barrier 2023-07-16 21:15:54,384 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-16 21:15:54,392 INFO [StoreFileOpener-rep_barrier-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a387d081595450199b46921f8b69aeb 2023-07-16 21:15:54,392 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/rep_barrier/7a387d081595450199b46921f8b69aeb 2023-07-16 21:15:54,393 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:54,393 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-16 21:15:54,394 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table 2023-07-16 21:15:54,394 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table 2023-07-16 21:15:54,394 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-16 21:15:54,405 INFO [StoreFileOpener-table-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e6b7923a5b4640e084b40f28aa6202c9 2023-07-16 21:15:54,405 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table/e6b7923a5b4640e084b40f28aa6202c9 2023-07-16 21:15:54,405 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:54,406 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740 2023-07-16 21:15:54,407 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740 2023-07-16 21:15:54,410 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-16 21:15:54,411 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-16 21:15:54,412 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=138; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11219490560, jitterRate=0.044896483421325684}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-16 21:15:54,412 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-16 21:15:54,413 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=111, masterSystemTime=1689542154322 2023-07-16 21:15:54,417 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-16 21:15:54,418 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-16 21:15:54,418 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34991,1689542153429, state=OPEN 2023-07-16 21:15:54,419 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-16 21:15:54,420 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-16 21:15:54,421 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-07-16 21:15:54,422 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34991,1689542153429 in 259 msec 2023-07-16 21:15:54,423 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-07-16 21:15:54,423 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 438 msec 2023-07-16 21:15:54,571 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33729,1689542152890] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:15:54,572 WARN [RS-EventLoopGroup-12-1] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase4.apache.org/172.31.14.131:45229 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:45229 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-16 21:15:54,573 DEBUG [RS-EventLoopGroup-12-1] ipc.FailedServers(52): Added failed server with address jenkins-hbase4.apache.org/172.31.14.131:45229 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:45229 2023-07-16 21:15:54,677 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33729,1689542152890] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:45229 this server is in the failed servers list 2023-07-16 21:15:54,882 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33729,1689542152890] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:45229 this server is in the failed servers list 2023-07-16 21:15:55,188 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33729,1689542152890] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:45229 this server is in the failed servers list 2023-07-16 21:15:55,640 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=3; waited=1653ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1502ms 2023-07-16 21:15:55,695 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33729,1689542152890] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:45229 this server is in the failed servers list 2023-07-16 21:15:56,708 WARN [RS-EventLoopGroup-12-1] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase4.apache.org/172.31.14.131:45229 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:45229 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-16 21:15:56,709 DEBUG [RS-EventLoopGroup-12-1] ipc.FailedServers(52): Added failed server with address jenkins-hbase4.apache.org/172.31.14.131:45229 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:45229 2023-07-16 21:15:57,143 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=3; waited=3156ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-16 21:15:58,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=4513ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-16 21:15:58,500 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-16 21:15:58,503 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=73c0efa65c224047ddc5ff1870c16b5e, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,45229,1689542134233, regionLocation=jenkins-hbase4.apache.org,45229,1689542134233, openSeqNum=2 2023-07-16 21:15:58,504 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=82d35ab01095f590b40a2bc17e2059db, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,45229,1689542134233, regionLocation=jenkins-hbase4.apache.org,45229,1689542134233, openSeqNum=2 2023-07-16 21:15:58,504 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-16 21:15:58,504 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689542218504 2023-07-16 21:15:58,504 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689542278504 2023-07-16 21:15:58,504 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-16 21:15:58,519 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,36993,1689542134044 had 1 regions 2023-07-16 21:15:58,519 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,33511,1689542134438 had 0 regions 2023-07-16 21:15:58,519 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,41213,1689542138767 had 0 regions 2023-07-16 21:15:58,519 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,45229,1689542134233 had 2 regions 2023-07-16 21:15:58,520 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33729,1689542152890-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:58,520 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33729,1689542152890-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:58,520 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33729,1689542152890-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:58,520 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33729, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:58,520 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:58,520 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=108, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,33511,1689542134438, splitWal=true, meta=false, isMeta: false 2023-07-16 21:15:58,520 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=107, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45229,1689542134233, splitWal=true, meta=false, isMeta: false 2023-07-16 21:15:58,520 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=106, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,41213,1689542138767, splitWal=true, meta=false, isMeta: false 2023-07-16 21:15:58,521 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=109, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,36993,1689542134044, splitWal=true, meta=true, isMeta: false 2023-07-16 21:15:58,521 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. is NOT online; state={73c0efa65c224047ddc5ff1870c16b5e state=OPEN, ts=1689542158503, server=jenkins-hbase4.apache.org,45229,1689542134233}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-16 21:15:58,522 DEBUG [PEWorker-2] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,33511,1689542134438-splitting 2023-07-16 21:15:58,524 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,33511,1689542134438-splitting dir is empty, no logs to split. 2023-07-16 21:15:58,524 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,33511,1689542134438 WAL count=0, meta=false 2023-07-16 21:15:58,524 DEBUG [PEWorker-4] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,45229,1689542134233-splitting 2023-07-16 21:15:58,525 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,45229,1689542134233-splitting dir is empty, no logs to split. 2023-07-16 21:15:58,525 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,45229,1689542134233 WAL count=0, meta=false 2023-07-16 21:15:58,525 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,41213,1689542138767-splitting 2023-07-16 21:15:58,526 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,41213,1689542138767-splitting dir is empty, no logs to split. 2023-07-16 21:15:58,526 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,41213,1689542138767 WAL count=0, meta=false 2023-07-16 21:15:58,526 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,45229,1689542134233/hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e., unknown_server=jenkins-hbase4.apache.org,45229,1689542134233/hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:58,527 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,36993,1689542134044-splitting dir is empty, no logs to split. 2023-07-16 21:15:58,528 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,36993,1689542134044 WAL count=0, meta=false 2023-07-16 21:15:58,529 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,33511,1689542134438-splitting dir is empty, no logs to split. 2023-07-16 21:15:58,529 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,33511,1689542134438 WAL count=0, meta=false 2023-07-16 21:15:58,529 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,33511,1689542134438 WAL splitting is done? wals=0, meta=false 2023-07-16 21:15:58,530 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,45229,1689542134233-splitting dir is empty, no logs to split. 2023-07-16 21:15:58,530 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,45229,1689542134233 WAL count=0, meta=false 2023-07-16 21:15:58,530 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,45229,1689542134233 WAL splitting is done? wals=0, meta=false 2023-07-16 21:15:58,531 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,41213,1689542138767-splitting dir is empty, no logs to split. 2023-07-16 21:15:58,531 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,41213,1689542138767 WAL count=0, meta=false 2023-07-16 21:15:58,531 INFO [PEWorker-2] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,33511,1689542134438 failed, ignore...File hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,33511,1689542134438-splitting does not exist. 2023-07-16 21:15:58,531 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,41213,1689542138767 WAL splitting is done? wals=0, meta=false 2023-07-16 21:15:58,533 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,36993,1689542134044-splitting dir is empty, no logs to split. 2023-07-16 21:15:58,533 INFO [PEWorker-4] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,45229,1689542134233 failed, ignore...File hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,45229,1689542134233-splitting does not exist. 2023-07-16 21:15:58,533 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,33511,1689542134438 after splitting done 2023-07-16 21:15:58,533 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,36993,1689542134044 WAL count=0, meta=false 2023-07-16 21:15:58,533 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,33511,1689542134438 from processing; numProcessing=3 2023-07-16 21:15:58,533 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,36993,1689542134044 WAL splitting is done? wals=0, meta=false 2023-07-16 21:15:58,534 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=112, ppid=107, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, ASSIGN}, {pid=113, ppid=107, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, ASSIGN}] 2023-07-16 21:15:58,535 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,33511,1689542134438, splitWal=true, meta=false in 4.6320 sec 2023-07-16 21:15:58,535 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,41213,1689542138767 failed, ignore...File hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,41213,1689542138767-splitting does not exist. 2023-07-16 21:15:58,535 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, ppid=107, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, ASSIGN 2023-07-16 21:15:58,535 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=107, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, ASSIGN 2023-07-16 21:15:58,539 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=112, ppid=107, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-16 21:15:58,539 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=107, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-16 21:15:58,539 DEBUG [jenkins-hbase4:33729] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-07-16 21:15:58,539 DEBUG [jenkins-hbase4:33729] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-16 21:15:58,539 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,41213,1689542138767 after splitting done 2023-07-16 21:15:58,540 DEBUG [jenkins-hbase4:33729] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-16 21:15:58,540 DEBUG [jenkins-hbase4:33729] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-16 21:15:58,540 DEBUG [jenkins-hbase4:33729] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-16 21:15:58,540 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,41213,1689542138767 from processing; numProcessing=2 2023-07-16 21:15:58,540 DEBUG [jenkins-hbase4:33729] balancer.BaseLoadBalancer$Cluster(378): Number of tables=2, number of hosts=1, number of racks=1 2023-07-16 21:15:58,542 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,36993,1689542134044 after splitting done 2023-07-16 21:15:58,542 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,36993,1689542134044 from processing; numProcessing=1 2023-07-16 21:15:58,542 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=112 updating hbase:meta row=73c0efa65c224047ddc5ff1870c16b5e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34345,1689542153062 2023-07-16 21:15:58,542 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689542158542"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542158542"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542158542"}]},"ts":"1689542158542"} 2023-07-16 21:15:58,543 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=82d35ab01095f590b40a2bc17e2059db, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:15:58,543 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1689542158543"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542158543"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542158543"}]},"ts":"1689542158543"} 2023-07-16 21:15:58,543 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=106, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,41213,1689542138767, splitWal=true, meta=false in 4.6480 sec 2023-07-16 21:15:58,544 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,36993,1689542134044, splitWal=true, meta=true in 4.6390 sec 2023-07-16 21:15:58,544 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=112, state=RUNNABLE; OpenRegionProcedure 73c0efa65c224047ddc5ff1870c16b5e, server=jenkins-hbase4.apache.org,34345,1689542153062}] 2023-07-16 21:15:58,545 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=113, state=RUNNABLE; OpenRegionProcedure 82d35ab01095f590b40a2bc17e2059db, server=jenkins-hbase4.apache.org,34991,1689542153429}] 2023-07-16 21:15:58,696 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34345,1689542153062 2023-07-16 21:15:58,696 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-16 21:15:58,698 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42458, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-16 21:15:58,703 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:58,703 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 82d35ab01095f590b40a2bc17e2059db, NAME => 'hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:15:58,703 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-16 21:15:58,703 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. service=MultiRowMutationService 2023-07-16 21:15:58,703 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:rsgroup successfully. 2023-07-16 21:15:58,704 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table rsgroup 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:58,704 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:58,704 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:15:58,704 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:58,704 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:58,704 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 73c0efa65c224047ddc5ff1870c16b5e, NAME => 'hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:15:58,704 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:58,704 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:58,704 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:58,704 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:58,705 INFO [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family m of region 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:58,706 INFO [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:58,707 DEBUG [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m 2023-07-16 21:15:58,707 DEBUG [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m 2023-07-16 21:15:58,707 INFO [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 82d35ab01095f590b40a2bc17e2059db columnFamilyName m 2023-07-16 21:15:58,707 DEBUG [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/info 2023-07-16 21:15:58,707 DEBUG [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/info 2023-07-16 21:15:58,708 INFO [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 73c0efa65c224047ddc5ff1870c16b5e columnFamilyName info 2023-07-16 21:15:58,718 INFO [StoreFileOpener-info-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7aca6aa2367450cb461f1ac0a602f75 2023-07-16 21:15:58,718 DEBUG [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/info/e7aca6aa2367450cb461f1ac0a602f75 2023-07-16 21:15:58,718 INFO [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] regionserver.HStore(310): Store=73c0efa65c224047ddc5ff1870c16b5e/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:58,719 INFO [StoreFileOpener-m-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d185255a3a8e4435a58d7c4f27f39d36 2023-07-16 21:15:58,720 DEBUG [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/d185255a3a8e4435a58d7c4f27f39d36 2023-07-16 21:15:58,720 INFO [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] regionserver.HStore(310): Store=82d35ab01095f590b40a2bc17e2059db/m, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:58,722 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:58,722 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:58,723 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:58,723 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:58,727 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:15:58,727 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:15:58,727 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 73c0efa65c224047ddc5ff1870c16b5e; next sequenceid=15; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11774464640, jitterRate=0.09658247232437134}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:58,728 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 73c0efa65c224047ddc5ff1870c16b5e: 2023-07-16 21:15:58,729 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 82d35ab01095f590b40a2bc17e2059db; next sequenceid=71; org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy@25a24ac9, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:15:58,729 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 82d35ab01095f590b40a2bc17e2059db: 2023-07-16 21:15:58,729 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e., pid=114, masterSystemTime=1689542158696 2023-07-16 21:15:58,732 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db., pid=115, masterSystemTime=1689542158697 2023-07-16 21:15:58,733 WARN [RS-EventLoopGroup-12-1] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase4.apache.org/172.31.14.131:45229 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:45229 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-16 21:15:58,734 DEBUG [RS-EventLoopGroup-12-1] ipc.FailedServers(52): Added failed server with address jenkins-hbase4.apache.org/172.31.14.131:45229 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:45229 2023-07-16 21:15:58,735 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33729,1689542152890] client.RpcRetryingCallerImpl(129): Call exception, tries=6, retries=46, started=4171 ms ago, cancelled=false, msg=Call to address=jenkins-hbase4.apache.org/172.31.14.131:45229 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:45229, details=row '\x00' on table 'hbase:rsgroup' at region=hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db., hostname=jenkins-hbase4.apache.org,45229,1689542134233, seqNum=2, see https://s.apache.org/timeout, exception=java.net.ConnectException: Call to address=jenkins-hbase4.apache.org/172.31.14.131:45229 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:45229 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:186) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:385) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:99) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:195) at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$300(NettyRpcConnection.java:76) at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:296) at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:287) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:45229 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-16 21:15:58,735 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:15:58,736 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:15:58,737 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=112 updating hbase:meta row=73c0efa65c224047ddc5ff1870c16b5e, regionState=OPEN, openSeqNum=15, regionLocation=jenkins-hbase4.apache.org,34345,1689542153062 2023-07-16 21:15:58,737 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689542158736"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542158736"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542158736"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542158736"}]},"ts":"1689542158736"} 2023-07-16 21:15:58,737 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:58,737 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:15:58,737 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=82d35ab01095f590b40a2bc17e2059db, regionState=OPEN, openSeqNum=71, regionLocation=jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:15:58,737 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1689542158737"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542158737"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542158737"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542158737"}]},"ts":"1689542158737"} 2023-07-16 21:15:58,740 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=112 2023-07-16 21:15:58,741 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=112, state=SUCCESS; OpenRegionProcedure 73c0efa65c224047ddc5ff1870c16b5e, server=jenkins-hbase4.apache.org,34345,1689542153062 in 194 msec 2023-07-16 21:15:58,742 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=113 2023-07-16 21:15:58,742 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=113, state=SUCCESS; OpenRegionProcedure 82d35ab01095f590b40a2bc17e2059db, server=jenkins-hbase4.apache.org,34991,1689542153429 in 194 msec 2023-07-16 21:15:58,742 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=112, ppid=107, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, ASSIGN in 207 msec 2023-07-16 21:15:58,744 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=107 2023-07-16 21:15:58,744 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,45229,1689542134233 after splitting done 2023-07-16 21:15:58,744 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=107, state=SUCCESS; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, ASSIGN in 208 msec 2023-07-16 21:15:58,744 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,45229,1689542134233 from processing; numProcessing=0 2023-07-16 21:15:58,745 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=107, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,45229,1689542134233, splitWal=true, meta=false in 4.8490 sec 2023-07-16 21:15:59,522 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/namespace 2023-07-16 21:15:59,530 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:15:59,531 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42464, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:15:59,542 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-16 21:15:59,545 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-07-16 21:15:59,545 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.909sec 2023-07-16 21:15:59,546 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(103): Quota table not found. Creating... 2023-07-16 21:15:59,546 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:quota', {NAME => 'q', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'u', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-16 21:15:59,547 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:quota 2023-07-16 21:15:59,547 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(107): Initializing quota support 2023-07-16 21:15:59,550 INFO [master/jenkins-hbase4:0:becomeActiveMaster] namespace.NamespaceStateManager(59): Namespace State Manager started. 2023-07-16 21:15:59,551 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_PRE_OPERATION 2023-07-16 21:15:59,556 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-16 21:15:59,556 INFO [master/jenkins-hbase4:0:becomeActiveMaster] namespace.NamespaceStateManager(222): Finished updating state of 2 namespaces. 2023-07-16 21:15:59,557 INFO [master/jenkins-hbase4:0:becomeActiveMaster] namespace.NamespaceAuditor(50): NamespaceAuditor started. 2023-07-16 21:15:59,558 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:15:59,558 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19 empty. 2023-07-16 21:15:59,559 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=QuotaObserverChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:59,559 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:15:59,559 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:quota regions 2023-07-16 21:15:59,559 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=QuotaObserverChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:15:59,559 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-16 21:15:59,559 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-16 21:15:59,559 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33729,1689542152890-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-16 21:15:59,559 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33729,1689542152890-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-16 21:15:59,560 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-16 21:15:59,581 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp/data/hbase/quota/.tabledesc/.tableinfo.0000000001 2023-07-16 21:15:59,583 INFO [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(7675): creating {ENCODED => 08c6c5dca9905cead995e8dedf4b2e19, NAME => 'hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:quota', {NAME => 'q', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'u', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.tmp 2023-07-16 21:15:59,595 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(866): Instantiated hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:59,595 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1604): Closing 08c6c5dca9905cead995e8dedf4b2e19, disabling compactions & flushes 2023-07-16 21:15:59,595 INFO [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1626): Closing region hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:15:59,595 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:15:59,595 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. after waiting 0 ms 2023-07-16 21:15:59,595 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:15:59,596 INFO [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1838): Closed hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:15:59,596 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1558): Region close journal for 08c6c5dca9905cead995e8dedf4b2e19: 2023-07-16 21:15:59,598 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_ADD_TO_META 2023-07-16 21:15:59,599 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1689542159598"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542159598"}]},"ts":"1689542159598"} 2023-07-16 21:15:59,600 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-16 21:15:59,601 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-16 21:15:59,601 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:quota","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542159601"}]},"ts":"1689542159601"} 2023-07-16 21:15:59,602 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:quota, state=ENABLING in hbase:meta 2023-07-16 21:15:59,606 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-16 21:15:59,606 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-16 21:15:59,606 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-16 21:15:59,606 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-16 21:15:59,606 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-16 21:15:59,606 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:quota, region=08c6c5dca9905cead995e8dedf4b2e19, ASSIGN}] 2023-07-16 21:15:59,608 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:quota, region=08c6c5dca9905cead995e8dedf4b2e19, ASSIGN 2023-07-16 21:15:59,609 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:quota, region=08c6c5dca9905cead995e8dedf4b2e19, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34991,1689542153429; forceNewPlan=false, retain=false 2023-07-16 21:15:59,620 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(139): Connect 0x26b50c0f to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:15:59,626 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3a49d51c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:15:59,629 DEBUG [hconnection-0xa0b6ecf-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:15:59,631 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57386, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:15:59,637 INFO [Listener at localhost/43605] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-16 21:15:59,637 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x26b50c0f to 127.0.0.1:63917 2023-07-16 21:15:59,637 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:15:59,639 INFO [Listener at localhost/43605] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:33729 after: jenkins-hbase4.apache.org:33729 2023-07-16 21:15:59,639 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(139): Connect 0x01e316fb to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:15:59,645 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@504f6d1e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:15:59,646 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:15:59,759 INFO [jenkins-hbase4:33729] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-16 21:15:59,760 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=08c6c5dca9905cead995e8dedf4b2e19, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:15:59,761 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1689542159760"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542159760"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542159760"}]},"ts":"1689542159760"} 2023-07-16 21:15:59,763 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=117, state=RUNNABLE; OpenRegionProcedure 08c6c5dca9905cead995e8dedf4b2e19, server=jenkins-hbase4.apache.org,34991,1689542153429}] 2023-07-16 21:15:59,869 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-16 21:15:59,918 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:15:59,918 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 08c6c5dca9905cead995e8dedf4b2e19, NAME => 'hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:15:59,919 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table quota 08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:15:59,919 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:15:59,919 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:15:59,919 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:15:59,921 INFO [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family q of region 08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:15:59,923 DEBUG [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19/q 2023-07-16 21:15:59,923 DEBUG [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19/q 2023-07-16 21:15:59,923 INFO [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 08c6c5dca9905cead995e8dedf4b2e19 columnFamilyName q 2023-07-16 21:15:59,928 INFO [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] regionserver.HStore(310): Store=08c6c5dca9905cead995e8dedf4b2e19/q, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:59,928 INFO [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family u of region 08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:15:59,929 DEBUG [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19/u 2023-07-16 21:15:59,929 DEBUG [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19/u 2023-07-16 21:15:59,930 INFO [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 08c6c5dca9905cead995e8dedf4b2e19 columnFamilyName u 2023-07-16 21:15:59,930 INFO [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] regionserver.HStore(310): Store=08c6c5dca9905cead995e8dedf4b2e19/u, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:15:59,931 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:15:59,932 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:15:59,934 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:quota descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-16 21:15:59,935 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:15:59,939 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-16 21:15:59,940 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 08c6c5dca9905cead995e8dedf4b2e19; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11137411840, jitterRate=0.03725230693817139}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-16 21:15:59,940 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 08c6c5dca9905cead995e8dedf4b2e19: 2023-07-16 21:15:59,947 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19., pid=118, masterSystemTime=1689542159914 2023-07-16 21:15:59,949 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:15:59,949 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:15:59,949 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=08c6c5dca9905cead995e8dedf4b2e19, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:15:59,949 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1689542159949"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542159949"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542159949"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542159949"}]},"ts":"1689542159949"} 2023-07-16 21:15:59,954 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=117 2023-07-16 21:15:59,954 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=117, state=SUCCESS; OpenRegionProcedure 08c6c5dca9905cead995e8dedf4b2e19, server=jenkins-hbase4.apache.org,34991,1689542153429 in 189 msec 2023-07-16 21:15:59,958 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-07-16 21:15:59,958 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=hbase:quota, region=08c6c5dca9905cead995e8dedf4b2e19, ASSIGN in 348 msec 2023-07-16 21:15:59,960 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-16 21:15:59,960 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:quota","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689542159960"}]},"ts":"1689542159960"} 2023-07-16 21:15:59,961 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:quota, state=ENABLED in hbase:meta 2023-07-16 21:15:59,964 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_POST_OPERATION 2023-07-16 21:15:59,965 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=hbase:quota in 418 msec 2023-07-16 21:16:00,109 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-16 21:16:00,110 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-16 21:16:00,111 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:rsgroup' 2023-07-16 21:16:00,111 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:quota' 2023-07-16 21:16:02,199 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver 2023-07-16 21:16:02,199 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver Metrics about HBase MasterObservers 2023-07-16 21:16:02,199 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.quotas.MasterQuotasObserver 2023-07-16 21:16:02,199 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.quotas.MasterQuotasObserver Metrics about HBase MasterObservers 2023-07-16 21:16:02,200 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-16 21:16:02,200 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-16 21:16:02,200 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint 2023-07-16 21:16:02,200 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint Metrics about HBase MasterObservers 2023-07-16 21:16:02,752 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33729,1689542152890] rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker(840): RSGroup table=hbase:rsgroup is online, refreshing cached information 2023-07-16 21:16:02,752 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33729,1689542152890] rsgroup.RSGroupInfoManagerImpl(534): Refreshing in Online mode. 2023-07-16 21:16:02,759 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33729,1689542152890] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:02,759 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33729,1689542152890] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:16:02,760 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33729,1689542152890] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:16:02,761 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rsgroup 2023-07-16 21:16:02,761 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33729,1689542152890] rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker(823): GroupBasedLoadBalancer is now online 2023-07-16 21:16:02,849 DEBUG [Listener at localhost/43605] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-16 21:16:02,851 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56838, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-16 21:16:02,853 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-16 21:16:02,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33729] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-16 21:16:02,854 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(139): Connect 0x3176a954 to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:16:02,862 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7882017e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:16:02,862 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=VerifyingRSGroupAdminClient connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:16:02,864 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [90,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:16:02,865 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): VerifyingRSGroupAdminClient0x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:16:02,865 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): VerifyingRSGroupAdminClient-0x1016ffd8078001b connected 2023-07-16 21:16:02,866 DEBUG [Listener at localhost/43605] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:16:02,867 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41914, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:16:02,874 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBasics(309): Shutting down cluster 2023-07-16 21:16:02,875 INFO [Listener at localhost/43605] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-16 21:16:02,875 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x01e316fb to 127.0.0.1:63917 2023-07-16 21:16:02,875 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:02,875 DEBUG [Listener at localhost/43605] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-16 21:16:02,875 DEBUG [Listener at localhost/43605] util.JVMClusterUtil(257): Found active master hash=1022879134, stopped=false 2023-07-16 21:16:02,875 DEBUG [Listener at localhost/43605] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint 2023-07-16 21:16:02,875 DEBUG [Listener at localhost/43605] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver 2023-07-16 21:16:02,875 DEBUG [Listener at localhost/43605] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.quotas.MasterQuotasObserver 2023-07-16 21:16:02,875 INFO [Listener at localhost/43605] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33729,1689542152890 2023-07-16 21:16:02,876 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-16 21:16:02,876 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-16 21:16:02,876 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-16 21:16:02,877 INFO [Listener at localhost/43605] procedure2.ProcedureExecutor(629): Stopping 2023-07-16 21:16:02,876 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41419-0x1016ffd80780012, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-16 21:16:02,877 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:16:02,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:16:02,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:16:02,878 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5c6ed03f to 127.0.0.1:63917 2023-07-16 21:16:02,878 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:02,878 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:16:02,878 INFO [Listener at localhost/43605] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34345,1689542153062' ***** 2023-07-16 21:16:02,878 INFO [Listener at localhost/43605] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-16 21:16:02,878 INFO [Listener at localhost/43605] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41419,1689542153256' ***** 2023-07-16 21:16:02,878 INFO [Listener at localhost/43605] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-16 21:16:02,878 INFO [Listener at localhost/43605] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34991,1689542153429' ***** 2023-07-16 21:16:02,878 INFO [Listener at localhost/43605] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-16 21:16:02,878 INFO [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:16:02,878 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41419-0x1016ffd80780012, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:16:02,878 INFO [RS:1;jenkins-hbase4:41419] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:16:02,878 INFO [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:16:02,888 INFO [RS:0;jenkins-hbase4:34345] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@15137874{regionserver,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:16:02,888 INFO [RS:1;jenkins-hbase4:41419] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@4294be60{regionserver,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:16:02,888 INFO [RS:2;jenkins-hbase4:34991] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@42067a36{regionserver,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:16:02,889 INFO [RS:0;jenkins-hbase4:34345] server.AbstractConnector(383): Stopped ServerConnector@33f7f8e1{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:16:02,889 INFO [RS:2;jenkins-hbase4:34991] server.AbstractConnector(383): Stopped ServerConnector@70dbea2d{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:16:02,889 INFO [RS:1;jenkins-hbase4:41419] server.AbstractConnector(383): Stopped ServerConnector@79a34b80{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:16:02,889 INFO [RS:2;jenkins-hbase4:34991] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:16:02,889 INFO [RS:0;jenkins-hbase4:34345] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:16:02,889 INFO [RS:1;jenkins-hbase4:41419] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:16:02,889 INFO [RS:2;jenkins-hbase4:34991] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@7de5dfd5{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:16:02,889 INFO [RS:0;jenkins-hbase4:34345] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@2d55c5ad{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:16:02,889 INFO [RS:2;jenkins-hbase4:34991] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@3e81019e{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:16:02,889 INFO [RS:1;jenkins-hbase4:41419] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@5ec7ab45{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:16:02,890 INFO [RS:0;jenkins-hbase4:34345] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@3e757193{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:16:02,890 INFO [RS:1;jenkins-hbase4:41419] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@7d525552{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:16:02,891 INFO [RS:1;jenkins-hbase4:41419] regionserver.HeapMemoryManager(220): Stopping 2023-07-16 21:16:02,891 INFO [RS:1;jenkins-hbase4:41419] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-16 21:16:02,891 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-16 21:16:02,891 INFO [RS:1;jenkins-hbase4:41419] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-16 21:16:02,891 INFO [RS:1;jenkins-hbase4:41419] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41419,1689542153256 2023-07-16 21:16:02,891 DEBUG [RS:1;jenkins-hbase4:41419] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x587ac046 to 127.0.0.1:63917 2023-07-16 21:16:02,892 DEBUG [RS:1;jenkins-hbase4:41419] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:02,892 INFO [RS:1;jenkins-hbase4:41419] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41419,1689542153256; all regions closed. 2023-07-16 21:16:02,892 INFO [RS:0;jenkins-hbase4:34345] regionserver.HeapMemoryManager(220): Stopping 2023-07-16 21:16:02,892 DEBUG [RS:1;jenkins-hbase4:41419] quotas.QuotaCache(100): Stopping QuotaRefresherChore chore. 2023-07-16 21:16:02,892 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-16 21:16:02,892 INFO [RS:2;jenkins-hbase4:34991] regionserver.HeapMemoryManager(220): Stopping 2023-07-16 21:16:02,892 INFO [RS:0;jenkins-hbase4:34345] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-16 21:16:02,893 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-16 21:16:02,893 INFO [RS:2;jenkins-hbase4:34991] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-16 21:16:02,893 INFO [RS:0;jenkins-hbase4:34345] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-16 21:16:02,893 INFO [RS:2;jenkins-hbase4:34991] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-16 21:16:02,893 INFO [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(3305): Received CLOSE for 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:02,893 INFO [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(3305): Received CLOSE for 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:02,894 INFO [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34345,1689542153062 2023-07-16 21:16:02,894 DEBUG [RS:0;jenkins-hbase4:34345] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2a68c75d to 127.0.0.1:63917 2023-07-16 21:16:02,894 DEBUG [RS:0;jenkins-hbase4:34345] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:02,894 INFO [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(3305): Received CLOSE for 08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:16:02,894 INFO [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-16 21:16:02,894 INFO [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:16:02,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 73c0efa65c224047ddc5ff1870c16b5e, disabling compactions & flushes 2023-07-16 21:16:02,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 82d35ab01095f590b40a2bc17e2059db, disabling compactions & flushes 2023-07-16 21:16:02,894 DEBUG [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(1478): Online Regions={73c0efa65c224047ddc5ff1870c16b5e=hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.} 2023-07-16 21:16:02,895 DEBUG [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(1504): Waiting on 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:02,895 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:02,895 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:02,894 DEBUG [RS:2;jenkins-hbase4:34991] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x13afa02a to 127.0.0.1:63917 2023-07-16 21:16:02,895 DEBUG [RS:2;jenkins-hbase4:34991] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:02,895 INFO [RS:2;jenkins-hbase4:34991] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-16 21:16:02,895 INFO [RS:2;jenkins-hbase4:34991] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-16 21:16:02,895 INFO [RS:2;jenkins-hbase4:34991] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-16 21:16:02,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:02,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:02,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. after waiting 0 ms 2023-07-16 21:16:02,897 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:02,895 INFO [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-16 21:16:02,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. after waiting 0 ms 2023-07-16 21:16:02,898 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:02,898 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 82d35ab01095f590b40a2bc17e2059db 1/1 column families, dataSize=242 B heapSize=648 B 2023-07-16 21:16:02,898 INFO [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-07-16 21:16:02,899 DEBUG [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, 82d35ab01095f590b40a2bc17e2059db=hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db., 08c6c5dca9905cead995e8dedf4b2e19=hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19.} 2023-07-16 21:16:02,899 DEBUG [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(1504): Waiting on 08c6c5dca9905cead995e8dedf4b2e19, 1588230740, 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:02,899 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-16 21:16:02,899 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-16 21:16:02,899 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-16 21:16:02,900 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-16 21:16:02,900 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-16 21:16:02,900 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.05 KB heapSize=5.87 KB 2023-07-16 21:16:02,903 DEBUG [RS:1;jenkins-hbase4:41419] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs 2023-07-16 21:16:02,904 INFO [RS:1;jenkins-hbase4:41419] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41419%2C1689542153256:(num 1689542154301) 2023-07-16 21:16:02,904 DEBUG [RS:1;jenkins-hbase4:41419] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:02,904 INFO [RS:1;jenkins-hbase4:41419] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:02,904 INFO [RS:1;jenkins-hbase4:41419] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-16 21:16:02,904 INFO [RS:1;jenkins-hbase4:41419] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-16 21:16:02,904 INFO [RS:1;jenkins-hbase4:41419] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-16 21:16:02,904 INFO [RS:1;jenkins-hbase4:41419] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-16 21:16:02,904 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:16:02,905 INFO [RS:1;jenkins-hbase4:41419] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41419 2023-07-16 21:16:02,908 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:02,908 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41419,1689542153256 2023-07-16 21:16:02,908 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41419-0x1016ffd80780012, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41419,1689542153256 2023-07-16 21:16:02,909 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41419-0x1016ffd80780012, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:02,909 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:02,909 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41419,1689542153256] 2023-07-16 21:16:02,909 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41419,1689542153256; numProcessing=1 2023-07-16 21:16:02,910 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41419,1689542153256 already deleted, retry=false 2023-07-16 21:16:02,910 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41419,1689542153256 expired; onlineServers=2 2023-07-16 21:16:02,913 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41419,1689542153256 2023-07-16 21:16:02,913 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:02,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=14 2023-07-16 21:16:02,925 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:02,925 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 73c0efa65c224047ddc5ff1870c16b5e: 2023-07-16 21:16:02,925 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:02,938 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=242 B at sequenceid=74 (bloomFilter=true), to=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/.tmp/m/23d165da51744f1692de8df99d4f87ef 2023-07-16 21:16:02,940 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.97 KB at sequenceid=149 (bloomFilter=false), to=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp/info/ca6165e191fc4d699c6c92331ab746d5 2023-07-16 21:16:02,947 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:02,950 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/.tmp/m/23d165da51744f1692de8df99d4f87ef as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/23d165da51744f1692de8df99d4f87ef 2023-07-16 21:16:02,957 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/23d165da51744f1692de8df99d4f87ef, entries=2, sequenceid=74, filesize=5.0 K 2023-07-16 21:16:02,960 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~242 B/242, heapSize ~632 B/632, currentSize=0 B/0 for 82d35ab01095f590b40a2bc17e2059db in 62ms, sequenceid=74, compaction requested=false 2023-07-16 21:16:02,967 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:02,970 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=86 B at sequenceid=149 (bloomFilter=false), to=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp/table/39ca1cf509684b6ab736d32a3c68a7be 2023-07-16 21:16:02,971 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:02,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/recovered.edits/77.seqid, newMaxSeqId=77, maxSeqId=70 2023-07-16 21:16:02,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-16 21:16:02,973 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:02,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 82d35ab01095f590b40a2bc17e2059db: 2023-07-16 21:16:02,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:02,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 08c6c5dca9905cead995e8dedf4b2e19, disabling compactions & flushes 2023-07-16 21:16:02,973 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:16:02,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:16:02,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. after waiting 0 ms 2023-07-16 21:16:02,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:16:02,977 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-16 21:16:02,978 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:16:02,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 08c6c5dca9905cead995e8dedf4b2e19: 2023-07-16 21:16:02,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:16:02,979 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp/info/ca6165e191fc4d699c6c92331ab746d5 as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/ca6165e191fc4d699c6c92331ab746d5 2023-07-16 21:16:02,984 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/ca6165e191fc4d699c6c92331ab746d5, entries=26, sequenceid=149, filesize=7.7 K 2023-07-16 21:16:02,985 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp/table/39ca1cf509684b6ab736d32a3c68a7be as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table/39ca1cf509684b6ab736d32a3c68a7be 2023-07-16 21:16:02,990 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table/39ca1cf509684b6ab736d32a3c68a7be, entries=2, sequenceid=149, filesize=4.7 K 2023-07-16 21:16:02,991 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.05 KB/3126, heapSize ~5.59 KB/5720, currentSize=0 B/0 for 1588230740 in 91ms, sequenceid=149, compaction requested=false 2023-07-16 21:16:03,002 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/recovered.edits/152.seqid, newMaxSeqId=152, maxSeqId=137 2023-07-16 21:16:03,002 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-16 21:16:03,003 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-16 21:16:03,003 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-16 21:16:03,003 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-16 21:16:03,076 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41419-0x1016ffd80780012, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:03,076 INFO [RS:1;jenkins-hbase4:41419] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41419,1689542153256; zookeeper connection closed. 2023-07-16 21:16:03,076 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41419-0x1016ffd80780012, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:03,078 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@25e2dd9f] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@25e2dd9f 2023-07-16 21:16:03,095 INFO [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34345,1689542153062; all regions closed. 2023-07-16 21:16:03,095 DEBUG [RS:0;jenkins-hbase4:34345] quotas.QuotaCache(100): Stopping QuotaRefresherChore chore. 2023-07-16 21:16:03,099 INFO [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34991,1689542153429; all regions closed. 2023-07-16 21:16:03,099 DEBUG [RS:2;jenkins-hbase4:34991] quotas.QuotaCache(100): Stopping QuotaRefresherChore chore. 2023-07-16 21:16:03,102 DEBUG [RS:0;jenkins-hbase4:34345] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs 2023-07-16 21:16:03,103 INFO [RS:0;jenkins-hbase4:34345] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34345%2C1689542153062:(num 1689542154301) 2023-07-16 21:16:03,103 DEBUG [RS:0;jenkins-hbase4:34345] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:03,103 INFO [RS:0;jenkins-hbase4:34345] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:03,103 INFO [RS:0;jenkins-hbase4:34345] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-16 21:16:03,103 INFO [RS:0;jenkins-hbase4:34345] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-16 21:16:03,103 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:16:03,103 INFO [RS:0;jenkins-hbase4:34345] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-16 21:16:03,103 INFO [RS:0;jenkins-hbase4:34345] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-16 21:16:03,105 INFO [RS:0;jenkins-hbase4:34345] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34345 2023-07-16 21:16:03,106 DEBUG [RS:2;jenkins-hbase4:34991] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs 2023-07-16 21:16:03,106 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34345,1689542153062 2023-07-16 21:16:03,106 INFO [RS:2;jenkins-hbase4:34991] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34991%2C1689542153429.meta:.meta(num 1689542154341) 2023-07-16 21:16:03,106 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:03,106 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34345,1689542153062 2023-07-16 21:16:03,109 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34345,1689542153062] 2023-07-16 21:16:03,109 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34345,1689542153062; numProcessing=2 2023-07-16 21:16:03,110 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34345,1689542153062 already deleted, retry=false 2023-07-16 21:16:03,110 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34345,1689542153062 expired; onlineServers=1 2023-07-16 21:16:03,112 DEBUG [RS:2;jenkins-hbase4:34991] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs 2023-07-16 21:16:03,112 INFO [RS:2;jenkins-hbase4:34991] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34991%2C1689542153429:(num 1689542154307) 2023-07-16 21:16:03,112 DEBUG [RS:2;jenkins-hbase4:34991] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:03,112 INFO [RS:2;jenkins-hbase4:34991] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:03,112 INFO [RS:2;jenkins-hbase4:34991] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-16 21:16:03,113 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:16:03,113 INFO [RS:2;jenkins-hbase4:34991] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34991 2023-07-16 21:16:03,115 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34991,1689542153429 2023-07-16 21:16:03,115 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:03,116 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34991,1689542153429] 2023-07-16 21:16:03,116 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34991,1689542153429; numProcessing=3 2023-07-16 21:16:03,118 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34991,1689542153429 already deleted, retry=false 2023-07-16 21:16:03,118 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34991,1689542153429 expired; onlineServers=0 2023-07-16 21:16:03,118 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33729,1689542152890' ***** 2023-07-16 21:16:03,118 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-16 21:16:03,118 DEBUG [M:0;jenkins-hbase4:33729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@38c876d8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:16:03,118 INFO [M:0;jenkins-hbase4:33729] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:16:03,120 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-07-16 21:16:03,120 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:16:03,120 INFO [M:0;jenkins-hbase4:33729] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@34f8329c{master,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/master} 2023-07-16 21:16:03,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-16 21:16:03,120 INFO [M:0;jenkins-hbase4:33729] server.AbstractConnector(383): Stopped ServerConnector@16af4387{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:16:03,121 INFO [M:0;jenkins-hbase4:33729] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:16:03,121 INFO [M:0;jenkins-hbase4:33729] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@7812c246{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:16:03,121 INFO [M:0;jenkins-hbase4:33729] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@a8ea588{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:16:03,121 INFO [M:0;jenkins-hbase4:33729] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33729,1689542152890 2023-07-16 21:16:03,121 INFO [M:0;jenkins-hbase4:33729] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33729,1689542152890; all regions closed. 2023-07-16 21:16:03,121 DEBUG [M:0;jenkins-hbase4:33729] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:03,122 INFO [M:0;jenkins-hbase4:33729] master.HMaster(1491): Stopping master jetty server 2023-07-16 21:16:03,122 INFO [M:0;jenkins-hbase4:33729] server.AbstractConnector(383): Stopped ServerConnector@1faa878d{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:16:03,123 DEBUG [M:0;jenkins-hbase4:33729] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-16 21:16:03,123 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-16 21:16:03,123 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689542153974] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689542153974,5,FailOnTimeoutGroup] 2023-07-16 21:16:03,123 DEBUG [M:0;jenkins-hbase4:33729] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-16 21:16:03,123 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689542153983] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689542153983,5,FailOnTimeoutGroup] 2023-07-16 21:16:03,124 INFO [M:0;jenkins-hbase4:33729] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-16 21:16:03,124 INFO [M:0;jenkins-hbase4:33729] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-16 21:16:03,124 INFO [M:0;jenkins-hbase4:33729] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [ScheduledChore name=QuotaObserverChore, period=60000, unit=MILLISECONDS] on shutdown 2023-07-16 21:16:03,125 DEBUG [M:0;jenkins-hbase4:33729] master.HMaster(1512): Stopping service threads 2023-07-16 21:16:03,125 INFO [M:0;jenkins-hbase4:33729] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-16 21:16:03,125 ERROR [M:0;jenkins-hbase4:33729] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-16 21:16:03,125 INFO [M:0;jenkins-hbase4:33729] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-16 21:16:03,125 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-16 21:16:03,126 DEBUG [M:0;jenkins-hbase4:33729] zookeeper.ZKUtil(398): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-07-16 21:16:03,126 WARN [M:0;jenkins-hbase4:33729] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-16 21:16:03,126 INFO [M:0;jenkins-hbase4:33729] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-16 21:16:03,126 INFO [M:0;jenkins-hbase4:33729] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-16 21:16:03,127 DEBUG [M:0;jenkins-hbase4:33729] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-16 21:16:03,127 INFO [M:0;jenkins-hbase4:33729] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:16:03,127 DEBUG [M:0;jenkins-hbase4:33729] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:16:03,127 DEBUG [M:0;jenkins-hbase4:33729] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-16 21:16:03,127 DEBUG [M:0;jenkins-hbase4:33729] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:16:03,127 INFO [M:0;jenkins-hbase4:33729] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=44.95 KB heapSize=54.44 KB 2023-07-16 21:16:03,144 INFO [M:0;jenkins-hbase4:33729] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=44.95 KB at sequenceid=885 (bloomFilter=true), to=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/bbd08d40647a45fb89060ba1106f0955 2023-07-16 21:16:03,150 DEBUG [M:0;jenkins-hbase4:33729] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/bbd08d40647a45fb89060ba1106f0955 as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/bbd08d40647a45fb89060ba1106f0955 2023-07-16 21:16:03,155 INFO [M:0;jenkins-hbase4:33729] regionserver.HStore(1080): Added hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/bbd08d40647a45fb89060ba1106f0955, entries=13, sequenceid=885, filesize=7.2 K 2023-07-16 21:16:03,156 INFO [M:0;jenkins-hbase4:33729] regionserver.HRegion(2948): Finished flush of dataSize ~44.95 KB/46033, heapSize ~54.42 KB/55728, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 29ms, sequenceid=885, compaction requested=false 2023-07-16 21:16:03,158 INFO [M:0;jenkins-hbase4:33729] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:16:03,158 DEBUG [M:0;jenkins-hbase4:33729] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-16 21:16:03,163 INFO [M:0;jenkins-hbase4:33729] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-16 21:16:03,163 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:16:03,164 INFO [M:0;jenkins-hbase4:33729] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33729 2023-07-16 21:16:03,166 DEBUG [M:0;jenkins-hbase4:33729] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,33729,1689542152890 already deleted, retry=false 2023-07-16 21:16:03,577 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:03,578 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33729-0x1016ffd80780010, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:03,577 INFO [M:0;jenkins-hbase4:33729] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33729,1689542152890; zookeeper connection closed. 2023-07-16 21:16:03,678 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:03,678 INFO [RS:2;jenkins-hbase4:34991] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34991,1689542153429; zookeeper connection closed. 2023-07-16 21:16:03,678 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34991-0x1016ffd80780013, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:03,678 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@64929263] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@64929263 2023-07-16 21:16:03,778 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:03,778 INFO [RS:0;jenkins-hbase4:34345] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34345,1689542153062; zookeeper connection closed. 2023-07-16 21:16:03,778 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34345-0x1016ffd80780011, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:03,778 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@645fcc6c] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@645fcc6c 2023-07-16 21:16:03,779 INFO [Listener at localhost/43605] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-07-16 21:16:03,779 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBasics(311): Sleeping a bit 2023-07-16 21:16:05,422 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-16 21:16:05,780 INFO [Listener at localhost/43605] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:16:05,781 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:05,781 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:05,781 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:16:05,781 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:05,781 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:16:05,781 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:16:05,782 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33973 2023-07-16 21:16:05,782 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:16:05,783 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:16:05,784 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33973 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:16:05,788 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:339730x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:16:05,788 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:33973-0x1016ffd8078001c connected 2023-07-16 21:16:05,795 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-16 21:16:05,795 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:16:05,796 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:16:05,796 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33973 2023-07-16 21:16:05,796 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33973 2023-07-16 21:16:05,798 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33973 2023-07-16 21:16:05,800 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33973 2023-07-16 21:16:05,801 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33973 2023-07-16 21:16:05,803 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:16:05,803 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:16:05,803 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:16:05,803 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context master 2023-07-16 21:16:05,803 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:16:05,803 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:16:05,804 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:16:05,804 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 38591 2023-07-16 21:16:05,804 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:16:05,808 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:05,808 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@2bc95537{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:16:05,808 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:05,808 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@78851778{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:16:05,922 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:16:05,923 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:16:05,923 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:16:05,924 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-16 21:16:05,924 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:05,925 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@4d6821d7{master,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-38591-hbase-server-2_4_18-SNAPSHOT_jar-_-any-4846029105462127205/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/master} 2023-07-16 21:16:05,926 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@14c4c302{HTTP/1.1, (http/1.1)}{0.0.0.0:38591} 2023-07-16 21:16:05,926 INFO [Listener at localhost/43605] server.Server(415): Started @39791ms 2023-07-16 21:16:05,927 INFO [Listener at localhost/43605] master.HMaster(444): hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb, hbase.cluster.distributed=false 2023-07-16 21:16:05,928 DEBUG [pool-512-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: INIT 2023-07-16 21:16:05,940 INFO [Listener at localhost/43605] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:16:05,940 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:05,940 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:05,941 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:16:05,941 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:05,941 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:16:05,941 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:16:05,941 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44109 2023-07-16 21:16:05,942 INFO [Listener at localhost/43605] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-16 21:16:05,943 DEBUG [Listener at localhost/43605] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-16 21:16:05,943 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:16:05,944 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:16:05,945 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:44109 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:16:05,949 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:441090x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:16:05,950 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:44109-0x1016ffd8078001d connected 2023-07-16 21:16:05,950 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-16 21:16:05,950 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:16:05,951 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:16:05,952 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44109 2023-07-16 21:16:05,953 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44109 2023-07-16 21:16:05,953 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44109 2023-07-16 21:16:05,954 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44109 2023-07-16 21:16:05,958 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44109 2023-07-16 21:16:05,960 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:16:05,960 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:16:05,960 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:16:05,961 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-16 21:16:05,961 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:16:05,961 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:16:05,961 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:16:05,961 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 45049 2023-07-16 21:16:05,962 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:16:05,963 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:05,963 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@43905abf{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:16:05,963 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:05,963 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@6711f9c9{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:16:06,074 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:16:06,075 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:16:06,075 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:16:06,076 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 600000ms 2023-07-16 21:16:06,076 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:06,077 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@65e3c6aa{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-45049-hbase-server-2_4_18-SNAPSHOT_jar-_-any-7068638562765980490/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:16:06,079 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@505610f7{HTTP/1.1, (http/1.1)}{0.0.0.0:45049} 2023-07-16 21:16:06,079 INFO [Listener at localhost/43605] server.Server(415): Started @39944ms 2023-07-16 21:16:06,091 INFO [Listener at localhost/43605] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:16:06,091 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:06,091 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:06,091 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:16:06,091 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:06,091 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:16:06,092 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:16:06,092 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40937 2023-07-16 21:16:06,093 INFO [Listener at localhost/43605] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-16 21:16:06,095 DEBUG [Listener at localhost/43605] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-16 21:16:06,096 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:16:06,096 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:16:06,097 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40937 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:16:06,102 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:409370x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:16:06,103 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:409370x0, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-16 21:16:06,103 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:40937-0x1016ffd8078001e connected 2023-07-16 21:16:06,104 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:16:06,104 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:16:06,108 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40937 2023-07-16 21:16:06,109 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40937 2023-07-16 21:16:06,109 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40937 2023-07-16 21:16:06,109 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40937 2023-07-16 21:16:06,110 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40937 2023-07-16 21:16:06,111 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:16:06,111 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:16:06,112 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:16:06,112 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-16 21:16:06,112 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:16:06,112 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:16:06,112 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:16:06,113 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 43323 2023-07-16 21:16:06,113 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:16:06,121 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:06,121 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@6e1f5678{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:16:06,121 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:06,121 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@f13ea19{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:16:06,236 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:16:06,236 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:16:06,236 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:16:06,237 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-16 21:16:06,237 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:06,238 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@41ecdee7{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-43323-hbase-server-2_4_18-SNAPSHOT_jar-_-any-1064876349747103668/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:16:06,240 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@4ede1e50{HTTP/1.1, (http/1.1)}{0.0.0.0:43323} 2023-07-16 21:16:06,240 INFO [Listener at localhost/43605] server.Server(415): Started @40105ms 2023-07-16 21:16:06,252 INFO [Listener at localhost/43605] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:16:06,252 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:06,252 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:06,252 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:16:06,252 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:06,252 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:16:06,252 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:16:06,253 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36613 2023-07-16 21:16:06,253 INFO [Listener at localhost/43605] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-16 21:16:06,255 DEBUG [Listener at localhost/43605] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-16 21:16:06,255 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:16:06,256 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:16:06,257 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36613 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:16:06,260 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:366130x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:16:06,261 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:366130x0, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-16 21:16:06,262 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36613-0x1016ffd8078001f connected 2023-07-16 21:16:06,262 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:16:06,263 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:16:06,269 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36613 2023-07-16 21:16:06,274 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36613 2023-07-16 21:16:06,274 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36613 2023-07-16 21:16:06,279 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36613 2023-07-16 21:16:06,279 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36613 2023-07-16 21:16:06,280 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:16:06,281 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:16:06,281 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:16:06,281 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-16 21:16:06,281 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:16:06,281 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:16:06,281 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:16:06,282 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 37073 2023-07-16 21:16:06,282 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:16:06,284 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:06,284 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@5f0f0e67{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:16:06,284 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:06,285 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@c3d8f8{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:16:06,411 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:16:06,411 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:16:06,412 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:16:06,412 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-16 21:16:06,412 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:06,413 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@2c606038{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-37073-hbase-server-2_4_18-SNAPSHOT_jar-_-any-2444491829938955668/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:16:06,415 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@21d758b1{HTTP/1.1, (http/1.1)}{0.0.0.0:37073} 2023-07-16 21:16:06,415 INFO [Listener at localhost/43605] server.Server(415): Started @40279ms 2023-07-16 21:16:06,418 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:16:06,426 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.AbstractConnector(333): Started ServerConnector@1933a51a{HTTP/1.1, (http/1.1)}{0.0.0.0:38513} 2023-07-16 21:16:06,426 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.Server(415): Started @40290ms 2023-07-16 21:16:06,426 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,33973,1689542165780 2023-07-16 21:16:06,428 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-16 21:16:06,428 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,33973,1689542165780 2023-07-16 21:16:06,429 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-16 21:16:06,429 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-16 21:16:06,429 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-16 21:16:06,429 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-16 21:16:06,431 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:16:06,430 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-16 21:16:06,433 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,33973,1689542165780 from backup master directory 2023-07-16 21:16:06,434 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-16 21:16:06,435 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,33973,1689542165780 2023-07-16 21:16:06,435 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-16 21:16:06,435 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:16:06,435 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,33973,1689542165780 2023-07-16 21:16:06,449 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:16:06,480 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x184da964 to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:16:06,486 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@e9bbaa9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:16:06,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-16 21:16:06,487 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-16 21:16:06,487 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:16:06,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,33729,1689542152890 to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,33729,1689542152890-dead as it is dead 2023-07-16 21:16:06,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,33729,1689542152890-dead/jenkins-hbase4.apache.org%2C33729%2C1689542152890.1689542153735 2023-07-16 21:16:06,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,33729,1689542152890-dead/jenkins-hbase4.apache.org%2C33729%2C1689542152890.1689542153735 after 0ms 2023-07-16 21:16:06,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,33729,1689542152890-dead/jenkins-hbase4.apache.org%2C33729%2C1689542152890.1689542153735 to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33729%2C1689542152890.1689542153735 2023-07-16 21:16:06,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,33729,1689542152890-dead 2023-07-16 21:16:06,496 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,33973,1689542165780 2023-07-16 21:16:06,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33973%2C1689542165780, suffix=, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,33973,1689542165780, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/oldWALs, maxLogs=10 2023-07-16 21:16:06,516 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:16:06,516 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:16:06,519 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:16:06,524 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/WALs/jenkins-hbase4.apache.org,33973,1689542165780/jenkins-hbase4.apache.org%2C33973%2C1689542165780.1689542166498 2023-07-16 21:16:06,526 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK], DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK], DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK]] 2023-07-16 21:16:06,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:16:06,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:16:06,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-16 21:16:06,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-16 21:16:06,532 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-16 21:16:06,533 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-16 21:16:06,534 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-16 21:16:06,543 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/4b615d5b292e4ecf9f959f15caa29d05 2023-07-16 21:16:06,550 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/bbd08d40647a45fb89060ba1106f0955 2023-07-16 21:16:06,550 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:16:06,550 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-16 21:16:06,551 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33729%2C1689542152890.1689542153735 2023-07-16 21:16:06,556 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 127, firstSequenceIdInLog=776, maxSequenceIdInLog=887, path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33729%2C1689542152890.1689542153735 2023-07-16 21:16:06,557 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33729%2C1689542152890.1689542153735 2023-07-16 21:16:06,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-16 21:16:06,571 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/887.seqid, newMaxSeqId=887, maxSeqId=774 2023-07-16 21:16:06,572 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=888; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11560979680, jitterRate=0.0767001360654831}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:16:06,572 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-16 21:16:06,573 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-16 21:16:06,574 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-16 21:16:06,574 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-16 21:16:06,574 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-16 21:16:06,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-16 21:16:06,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-16 21:16:06,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-16 21:16:06,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=6, state=SUCCESS; CreateTableProcedure table=hbase:rsgroup 2023-07-16 21:16:06,586 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=10, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-16 21:16:06,586 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=11, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-16 21:16:06,586 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,33295,1689542138167, splitWal=true, meta=false 2023-07-16 21:16:06,586 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=13, state=SUCCESS; ModifyNamespaceProcedure, namespace=default 2023-07-16 21:16:06,586 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=14, state=SUCCESS; CreateTableProcedure table=Group_testCreateAndAssign 2023-07-16 21:16:06,587 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=17, state=SUCCESS; DisableTableProcedure table=Group_testCreateAndAssign 2023-07-16 21:16:06,587 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=20, state=SUCCESS; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-16 21:16:06,588 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=21, state=SUCCESS; CreateTableProcedure table=Group_testCreateMultiRegion 2023-07-16 21:16:06,588 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=42, state=SUCCESS; DisableTableProcedure table=Group_testCreateMultiRegion 2023-07-16 21:16:06,588 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-16 21:16:06,588 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=64, state=SUCCESS; CreateNamespaceProcedure, namespace=Group_foo 2023-07-16 21:16:06,588 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=65, state=SUCCESS; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-16 21:16:06,588 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DisableTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-16 21:16:06,588 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=71, state=SUCCESS; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-16 21:16:06,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-16 21:16:06,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=73, state=SUCCESS; CreateTableProcedure table=Group_testCreateAndDrop 2023-07-16 21:16:06,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=76, state=SUCCESS; DisableTableProcedure table=Group_testCreateAndDrop 2023-07-16 21:16:06,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-16 21:16:06,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=80, state=SUCCESS; CreateTableProcedure table=Group_testCloneSnapshot 2023-07-16 21:16:06,590 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=83, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=EXCLUSIVE 2023-07-16 21:16:06,590 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=SHARED 2023-07-16 21:16:06,590 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CloneSnapshotProcedure (table=Group_testCloneSnapshot_clone snapshot=name: "Group_testCloneSnapshot_snap" table: "Group_testCloneSnapshot" creation_time: 1689542145702 type: FLUSH version: 2 ttl: 0 ) 2023-07-16 21:16:06,590 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; DisableTableProcedure table=Group_testCloneSnapshot 2023-07-16 21:16:06,590 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=91, state=SUCCESS; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-16 21:16:06,591 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=92, state=SUCCESS; DisableTableProcedure table=Group_testCloneSnapshot_clone 2023-07-16 21:16:06,591 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=95, state=SUCCESS; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-16 21:16:06,591 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; CreateNamespaceProcedure, namespace=Group_ns 2023-07-16 21:16:06,592 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=97, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-create-table:org.apache.hadoop.hbase.HBaseIOException: No online servers in the rsgroup appInfo which table Group_ns:testCreateWhenRsgroupNoOnlineServers belongs to; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:16:06,592 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=98, state=SUCCESS; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:16:06,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DisableTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:16:06,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=104, state=SUCCESS; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-16 21:16:06,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-16 21:16:06,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=106, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,41213,1689542138767, splitWal=true, meta=false 2023-07-16 21:16:06,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=107, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,45229,1689542134233, splitWal=true, meta=false 2023-07-16 21:16:06,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,33511,1689542134438, splitWal=true, meta=false 2023-07-16 21:16:06,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,36993,1689542134044, splitWal=true, meta=true 2023-07-16 21:16:06,594 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=116, state=SUCCESS; CreateTableProcedure table=hbase:quota 2023-07-16 21:16:06,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 18 msec 2023-07-16 21:16:06,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-16 21:16:06,595 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-16 21:16:06,595 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34991,1689542153429, table=hbase:meta, region=1588230740 2023-07-16 21:16:06,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 3 possibly 'live' servers, and 0 'splitting'. 2023-07-16 21:16:06,599 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34345,1689542153062 already deleted, retry=false 2023-07-16 21:16:06,599 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,34345,1689542153062 on jenkins-hbase4.apache.org,33973,1689542165780 2023-07-16 21:16:06,601 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,34345,1689542153062, splitWal=true, meta=false 2023-07-16 21:16:06,601 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=119 for jenkins-hbase4.apache.org,34345,1689542153062 (carryingMeta=false) jenkins-hbase4.apache.org,34345,1689542153062/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@68bfe50a[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-16 21:16:06,603 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41419,1689542153256 already deleted, retry=false 2023-07-16 21:16:06,603 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,41419,1689542153256 on jenkins-hbase4.apache.org,33973,1689542165780 2023-07-16 21:16:06,607 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=120, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,41419,1689542153256, splitWal=true, meta=false 2023-07-16 21:16:06,608 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=120 for jenkins-hbase4.apache.org,41419,1689542153256 (carryingMeta=false) jenkins-hbase4.apache.org,41419,1689542153256/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@3bbbc24d[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-16 21:16:06,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34991,1689542153429 already deleted, retry=false 2023-07-16 21:16:06,609 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,34991,1689542153429 on jenkins-hbase4.apache.org,33973,1689542165780 2023-07-16 21:16:06,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=121, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,34991,1689542153429, splitWal=true, meta=true 2023-07-16 21:16:06,615 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=121 for jenkins-hbase4.apache.org,34991,1689542153429 (carryingMeta=true) jenkins-hbase4.apache.org,34991,1689542153429/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@1004e821[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-16 21:16:06,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/balancer 2023-07-16 21:16:06,616 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-16 21:16:06,616 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-07-16 21:16:06,623 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-07-16 21:16:06,623 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-07-16 21:16:06,624 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-07-16 21:16:06,626 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-16 21:16:06,626 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-16 21:16:06,626 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-16 21:16:06,626 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-16 21:16:06,626 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:16:06,630 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,33973,1689542165780, sessionid=0x1016ffd8078001c, setting cluster-up flag (Was=false) 2023-07-16 21:16:06,632 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-07-16 21:16:06,633 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33973,1689542165780 2023-07-16 21:16:06,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-07-16 21:16:06,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33973,1689542165780 2023-07-16 21:16:06,637 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/.hbase-snapshot/.tmp 2023-07-16 21:16:06,639 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2930): Registered master coprocessor service: service=RSGroupAdminService 2023-07-16 21:16:06,639 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] rsgroup.RSGroupInfoManagerImpl(537): Refreshing in Offline mode. 2023-07-16 21:16:06,640 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] rsgroup.RSGroupInfoManagerImpl(511): Read ZK GroupInfo count:2 2023-07-16 21:16:06,643 INFO [master/jenkins-hbase4:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint loaded, priority=536870911. 2023-07-16 21:16:06,643 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:16:06,643 INFO [master/jenkins-hbase4:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver loaded, priority=536870912. 2023-07-16 21:16:06,648 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33973,1689542165780] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:16:06,649 WARN [RS-EventLoopGroup-16-3] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase4.apache.org/172.31.14.131:34991 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:34991 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-16 21:16:06,651 DEBUG [RS-EventLoopGroup-16-3] ipc.FailedServers(52): Added failed server with address jenkins-hbase4.apache.org/172.31.14.131:34991 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:34991 2023-07-16 21:16:06,659 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-16 21:16:06,659 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-16 21:16:06,659 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-16 21:16:06,659 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-16 21:16:06,659 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-16 21:16:06,659 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-16 21:16:06,659 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-16 21:16:06,659 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-16 21:16:06,659 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-16 21:16:06,659 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,659 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:16:06,659 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689542196664 2023-07-16 21:16:06,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-16 21:16:06,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-16 21:16:06,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-16 21:16:06,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-16 21:16:06,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-16 21:16:06,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-16 21:16:06,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,669 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-16 21:16:06,669 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-16 21:16:06,669 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-16 21:16:06,670 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-16 21:16:06,670 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-16 21:16:06,671 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689542166670,5,FailOnTimeoutGroup] 2023-07-16 21:16:06,671 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41419,1689542153256; numProcessing=1 2023-07-16 21:16:06,671 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34991,1689542153429; numProcessing=2 2023-07-16 21:16:06,671 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=120, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,41419,1689542153256, splitWal=true, meta=false 2023-07-16 21:16:06,671 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34345,1689542153062; numProcessing=3 2023-07-16 21:16:06,671 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=121, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34991,1689542153429, splitWal=true, meta=true 2023-07-16 21:16:06,671 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=119, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34345,1689542153062, splitWal=true, meta=false 2023-07-16 21:16:06,671 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=121, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34991,1689542153429, splitWal=true, meta=true, isMeta: true 2023-07-16 21:16:06,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689542166671,5,FailOnTimeoutGroup] 2023-07-16 21:16:06,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-16 21:16:06,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689542166672, completionTime=-1 2023-07-16 21:16:06,673 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-16 21:16:06,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-16 21:16:06,674 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34991,1689542153429-splitting 2023-07-16 21:16:06,675 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34991,1689542153429-splitting dir is empty, no logs to split. 2023-07-16 21:16:06,675 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,34991,1689542153429 WAL count=0, meta=true 2023-07-16 21:16:06,677 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34991,1689542153429-splitting dir is empty, no logs to split. 2023-07-16 21:16:06,677 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,34991,1689542153429 WAL count=0, meta=true 2023-07-16 21:16:06,677 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34991,1689542153429 WAL splitting is done? wals=0, meta=true 2023-07-16 21:16:06,679 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-16 21:16:06,680 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=122, ppid=121, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-16 21:16:06,682 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=122, ppid=121, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-16 21:16:06,717 INFO [RS:1;jenkins-hbase4:40937] regionserver.HRegionServer(951): ClusterId : bc8d5a8a-d305-489f-bc1d-b95b98296a6c 2023-07-16 21:16:06,717 INFO [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(951): ClusterId : bc8d5a8a-d305-489f-bc1d-b95b98296a6c 2023-07-16 21:16:06,718 INFO [RS:2;jenkins-hbase4:36613] regionserver.HRegionServer(951): ClusterId : bc8d5a8a-d305-489f-bc1d-b95b98296a6c 2023-07-16 21:16:06,722 DEBUG [RS:1;jenkins-hbase4:40937] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-16 21:16:06,722 DEBUG [RS:0;jenkins-hbase4:44109] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-16 21:16:06,723 DEBUG [RS:2;jenkins-hbase4:36613] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-16 21:16:06,727 DEBUG [RS:1;jenkins-hbase4:40937] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-16 21:16:06,727 DEBUG [RS:1;jenkins-hbase4:40937] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-16 21:16:06,727 DEBUG [RS:0;jenkins-hbase4:44109] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-16 21:16:06,727 DEBUG [RS:0;jenkins-hbase4:44109] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-16 21:16:06,727 DEBUG [RS:2;jenkins-hbase4:36613] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-16 21:16:06,727 DEBUG [RS:2;jenkins-hbase4:36613] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-16 21:16:06,732 DEBUG [RS:1;jenkins-hbase4:40937] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-16 21:16:06,732 DEBUG [RS:2;jenkins-hbase4:36613] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-16 21:16:06,733 DEBUG [RS:0;jenkins-hbase4:44109] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-16 21:16:06,741 DEBUG [RS:1;jenkins-hbase4:40937] zookeeper.ReadOnlyZKClient(139): Connect 0x792f238d to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:16:06,741 DEBUG [RS:0;jenkins-hbase4:44109] zookeeper.ReadOnlyZKClient(139): Connect 0x3716accb to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:16:06,741 DEBUG [RS:2;jenkins-hbase4:36613] zookeeper.ReadOnlyZKClient(139): Connect 0x7ba085ac to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:16:06,754 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33973,1689542165780] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:34991 this server is in the failed servers list 2023-07-16 21:16:06,762 DEBUG [RS:1;jenkins-hbase4:40937] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@77e8d2d8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:16:06,763 DEBUG [RS:0;jenkins-hbase4:44109] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@39d14459, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:16:06,763 DEBUG [RS:1;jenkins-hbase4:40937] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@73d748fd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:16:06,763 DEBUG [RS:2;jenkins-hbase4:36613] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@756764a8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:16:06,763 DEBUG [RS:0;jenkins-hbase4:44109] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3deec7bb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:16:06,763 DEBUG [RS:2;jenkins-hbase4:36613] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7158eedc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:16:06,771 DEBUG [RS:2;jenkins-hbase4:36613] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:36613 2023-07-16 21:16:06,771 INFO [RS:2;jenkins-hbase4:36613] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-16 21:16:06,772 INFO [RS:2;jenkins-hbase4:36613] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-16 21:16:06,772 DEBUG [RS:2;jenkins-hbase4:36613] regionserver.HRegionServer(1022): About to register with Master. 2023-07-16 21:16:06,772 INFO [RS:2;jenkins-hbase4:36613] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33973,1689542165780 with isa=jenkins-hbase4.apache.org/172.31.14.131:36613, startcode=1689542166251 2023-07-16 21:16:06,772 DEBUG [RS:2;jenkins-hbase4:36613] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-16 21:16:06,774 INFO [RS-EventLoopGroup-13-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42385, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.10 (auth:SIMPLE), service=RegionServerStatusService 2023-07-16 21:16:06,775 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33973] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:06,775 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:16:06,776 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 1 2023-07-16 21:16:06,776 DEBUG [RS:2;jenkins-hbase4:36613] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb 2023-07-16 21:16:06,776 DEBUG [RS:2;jenkins-hbase4:36613] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35261 2023-07-16 21:16:06,776 DEBUG [RS:2;jenkins-hbase4:36613] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=38591 2023-07-16 21:16:06,776 DEBUG [RS:0;jenkins-hbase4:44109] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:44109 2023-07-16 21:16:06,777 INFO [RS:0;jenkins-hbase4:44109] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-16 21:16:06,777 INFO [RS:0;jenkins-hbase4:44109] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-16 21:16:06,777 DEBUG [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(1022): About to register with Master. 2023-07-16 21:16:06,777 DEBUG [RS:1;jenkins-hbase4:40937] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:40937 2023-07-16 21:16:06,777 INFO [RS:1;jenkins-hbase4:40937] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-16 21:16:06,777 INFO [RS:1;jenkins-hbase4:40937] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-16 21:16:06,777 DEBUG [RS:1;jenkins-hbase4:40937] regionserver.HRegionServer(1022): About to register with Master. 2023-07-16 21:16:06,778 INFO [RS:1;jenkins-hbase4:40937] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33973,1689542165780 with isa=jenkins-hbase4.apache.org/172.31.14.131:40937, startcode=1689542166090 2023-07-16 21:16:06,778 DEBUG [RS:1;jenkins-hbase4:40937] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-16 21:16:06,778 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:06,778 DEBUG [RS:2;jenkins-hbase4:36613] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:06,778 WARN [RS:2;jenkins-hbase4:36613] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:16:06,778 INFO [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33973,1689542165780 with isa=jenkins-hbase4.apache.org/172.31.14.131:44109, startcode=1689542165940 2023-07-16 21:16:06,778 DEBUG [RS:0;jenkins-hbase4:44109] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-16 21:16:06,778 INFO [RS:2;jenkins-hbase4:36613] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:16:06,779 DEBUG [RS:2;jenkins-hbase4:36613] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:06,780 INFO [RS-EventLoopGroup-13-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53645, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.9 (auth:SIMPLE), service=RegionServerStatusService 2023-07-16 21:16:06,781 INFO [RS-EventLoopGroup-13-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57307, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=RegionServerStatusService 2023-07-16 21:16:06,781 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36613,1689542166251] 2023-07-16 21:16:06,781 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33973] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:06,781 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:16:06,781 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 2 2023-07-16 21:16:06,781 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33973] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:06,781 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:16:06,781 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 3 2023-07-16 21:16:06,781 DEBUG [RS:1;jenkins-hbase4:40937] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb 2023-07-16 21:16:06,782 DEBUG [RS:1;jenkins-hbase4:40937] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35261 2023-07-16 21:16:06,782 DEBUG [RS:1;jenkins-hbase4:40937] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=38591 2023-07-16 21:16:06,782 DEBUG [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb 2023-07-16 21:16:06,782 DEBUG [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35261 2023-07-16 21:16:06,782 DEBUG [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=38591 2023-07-16 21:16:06,784 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:06,785 DEBUG [RS:1;jenkins-hbase4:40937] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:06,785 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,40937,1689542166090] 2023-07-16 21:16:06,786 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,44109,1689542165940] 2023-07-16 21:16:06,785 WARN [RS:1;jenkins-hbase4:40937] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:16:06,786 INFO [RS:1;jenkins-hbase4:40937] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:16:06,786 DEBUG [RS:1;jenkins-hbase4:40937] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:06,787 DEBUG [RS:0;jenkins-hbase4:44109] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:06,787 WARN [RS:0;jenkins-hbase4:44109] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:16:06,787 INFO [RS:0;jenkins-hbase4:44109] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:16:06,787 DEBUG [RS:2;jenkins-hbase4:36613] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:06,788 DEBUG [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:06,788 DEBUG [RS:2;jenkins-hbase4:36613] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:06,788 DEBUG [RS:2;jenkins-hbase4:36613] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:06,789 DEBUG [RS:2;jenkins-hbase4:36613] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-16 21:16:06,789 INFO [RS:2;jenkins-hbase4:36613] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-16 21:16:06,795 INFO [RS:2;jenkins-hbase4:36613] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-16 21:16:06,796 INFO [RS:2;jenkins-hbase4:36613] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-16 21:16:06,796 INFO [RS:2;jenkins-hbase4:36613] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,797 DEBUG [RS:1;jenkins-hbase4:40937] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:06,798 DEBUG [RS:1;jenkins-hbase4:40937] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:06,798 DEBUG [RS:1;jenkins-hbase4:40937] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:06,799 DEBUG [RS:1;jenkins-hbase4:40937] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-16 21:16:06,799 INFO [RS:1;jenkins-hbase4:40937] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-16 21:16:06,807 INFO [RS:2;jenkins-hbase4:36613] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-16 21:16:06,807 INFO [RS:1;jenkins-hbase4:40937] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-16 21:16:06,811 INFO [RS:1;jenkins-hbase4:40937] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-16 21:16:06,811 INFO [RS:1;jenkins-hbase4:40937] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,812 INFO [RS:1;jenkins-hbase4:40937] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-16 21:16:06,813 INFO [RS:2;jenkins-hbase4:36613] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,813 DEBUG [RS:2;jenkins-hbase4:36613] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,813 DEBUG [RS:2;jenkins-hbase4:36613] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,813 DEBUG [RS:2;jenkins-hbase4:36613] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,813 DEBUG [RS:2;jenkins-hbase4:36613] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,813 DEBUG [RS:2;jenkins-hbase4:36613] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,813 INFO [RS:1;jenkins-hbase4:40937] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,813 DEBUG [RS:2;jenkins-hbase4:36613] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:16:06,813 DEBUG [RS:1;jenkins-hbase4:40937] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,813 DEBUG [RS:2;jenkins-hbase4:36613] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,814 DEBUG [RS:1;jenkins-hbase4:40937] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,814 DEBUG [RS:2;jenkins-hbase4:36613] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,814 DEBUG [RS:1;jenkins-hbase4:40937] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,814 DEBUG [RS:0;jenkins-hbase4:44109] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:06,814 DEBUG [RS:1;jenkins-hbase4:40937] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,814 DEBUG [RS:2;jenkins-hbase4:36613] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,814 DEBUG [RS:1;jenkins-hbase4:40937] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,814 DEBUG [RS:2;jenkins-hbase4:36613] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,814 DEBUG [RS:1;jenkins-hbase4:40937] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:16:06,814 DEBUG [RS:1;jenkins-hbase4:40937] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,814 DEBUG [RS:1;jenkins-hbase4:40937] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,814 DEBUG [RS:1;jenkins-hbase4:40937] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,814 DEBUG [RS:1;jenkins-hbase4:40937] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,815 DEBUG [RS:0;jenkins-hbase4:44109] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:06,815 DEBUG [RS:0;jenkins-hbase4:44109] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:06,816 DEBUG [RS:0;jenkins-hbase4:44109] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-16 21:16:06,816 INFO [RS:0;jenkins-hbase4:44109] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-16 21:16:06,823 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=3; waited=150ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-16 21:16:06,825 INFO [RS:0;jenkins-hbase4:44109] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-16 21:16:06,825 INFO [RS:0;jenkins-hbase4:44109] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-16 21:16:06,825 INFO [RS:0;jenkins-hbase4:44109] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,826 INFO [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-16 21:16:06,831 INFO [RS:2;jenkins-hbase4:36613] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,831 INFO [RS:2;jenkins-hbase4:36613] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,831 INFO [RS:2;jenkins-hbase4:36613] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,832 DEBUG [jenkins-hbase4:33973] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-07-16 21:16:06,832 DEBUG [jenkins-hbase4:33973] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-16 21:16:06,832 DEBUG [jenkins-hbase4:33973] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-16 21:16:06,832 DEBUG [jenkins-hbase4:33973] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-16 21:16:06,832 DEBUG [jenkins-hbase4:33973] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-16 21:16:06,832 DEBUG [jenkins-hbase4:33973] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-16 21:16:06,843 INFO [RS:1;jenkins-hbase4:40937] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,843 INFO [RS:1;jenkins-hbase4:40937] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,843 INFO [RS:1;jenkins-hbase4:40937] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,843 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40937,1689542166090, state=OPENING 2023-07-16 21:16:06,844 INFO [RS:0;jenkins-hbase4:44109] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,844 DEBUG [RS:0;jenkins-hbase4:44109] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,844 DEBUG [RS:0;jenkins-hbase4:44109] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,844 DEBUG [RS:0;jenkins-hbase4:44109] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,844 DEBUG [RS:0;jenkins-hbase4:44109] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,844 DEBUG [RS:0;jenkins-hbase4:44109] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,844 DEBUG [RS:0;jenkins-hbase4:44109] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:16:06,844 DEBUG [RS:0;jenkins-hbase4:44109] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,844 DEBUG [RS:0;jenkins-hbase4:44109] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,844 DEBUG [RS:0;jenkins-hbase4:44109] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,844 DEBUG [RS:0;jenkins-hbase4:44109] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:06,846 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-16 21:16:06,847 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-16 21:16:06,847 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=122, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40937,1689542166090}] 2023-07-16 21:16:06,862 INFO [RS:1;jenkins-hbase4:40937] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-16 21:16:06,862 INFO [RS:1;jenkins-hbase4:40937] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40937,1689542166090-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,863 INFO [RS:0;jenkins-hbase4:44109] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,863 INFO [RS:0;jenkins-hbase4:44109] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,863 INFO [RS:0;jenkins-hbase4:44109] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,865 INFO [RS:2;jenkins-hbase4:36613] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-16 21:16:06,865 INFO [RS:2;jenkins-hbase4:36613] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36613,1689542166251-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,884 INFO [RS:1;jenkins-hbase4:40937] regionserver.Replication(203): jenkins-hbase4.apache.org,40937,1689542166090 started 2023-07-16 21:16:06,884 INFO [RS:1;jenkins-hbase4:40937] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,40937,1689542166090, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:40937, sessionid=0x1016ffd8078001e 2023-07-16 21:16:06,885 DEBUG [RS:1;jenkins-hbase4:40937] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-16 21:16:06,885 DEBUG [RS:1;jenkins-hbase4:40937] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:06,885 DEBUG [RS:1;jenkins-hbase4:40937] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40937,1689542166090' 2023-07-16 21:16:06,885 DEBUG [RS:1;jenkins-hbase4:40937] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-16 21:16:06,888 DEBUG [RS:1;jenkins-hbase4:40937] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-16 21:16:06,889 DEBUG [RS:1;jenkins-hbase4:40937] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-16 21:16:06,889 DEBUG [RS:1;jenkins-hbase4:40937] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-16 21:16:06,889 DEBUG [RS:1;jenkins-hbase4:40937] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:06,889 DEBUG [RS:1;jenkins-hbase4:40937] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40937,1689542166090' 2023-07-16 21:16:06,889 DEBUG [RS:1;jenkins-hbase4:40937] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:16:06,889 INFO [RS:0;jenkins-hbase4:44109] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-16 21:16:06,890 INFO [RS:0;jenkins-hbase4:44109] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44109,1689542165940-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:06,890 DEBUG [RS:1;jenkins-hbase4:40937] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:16:06,896 DEBUG [RS:1;jenkins-hbase4:40937] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-16 21:16:06,897 INFO [RS:1;jenkins-hbase4:40937] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-16 21:16:06,897 INFO [RS:1;jenkins-hbase4:40937] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-16 21:16:06,901 INFO [RS:2;jenkins-hbase4:36613] regionserver.Replication(203): jenkins-hbase4.apache.org,36613,1689542166251 started 2023-07-16 21:16:06,901 INFO [RS:2;jenkins-hbase4:36613] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,36613,1689542166251, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36613, sessionid=0x1016ffd8078001f 2023-07-16 21:16:06,901 DEBUG [RS:2;jenkins-hbase4:36613] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-16 21:16:06,901 DEBUG [RS:2;jenkins-hbase4:36613] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:06,901 DEBUG [RS:2;jenkins-hbase4:36613] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36613,1689542166251' 2023-07-16 21:16:06,901 DEBUG [RS:2;jenkins-hbase4:36613] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-16 21:16:06,904 DEBUG [RS:2;jenkins-hbase4:36613] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-16 21:16:06,905 DEBUG [RS:2;jenkins-hbase4:36613] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-16 21:16:06,905 DEBUG [RS:2;jenkins-hbase4:36613] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-16 21:16:06,905 DEBUG [RS:2;jenkins-hbase4:36613] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:06,905 DEBUG [RS:2;jenkins-hbase4:36613] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36613,1689542166251' 2023-07-16 21:16:06,905 DEBUG [RS:2;jenkins-hbase4:36613] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:16:06,905 DEBUG [RS:2;jenkins-hbase4:36613] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:16:06,908 DEBUG [RS:2;jenkins-hbase4:36613] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-16 21:16:06,908 INFO [RS:2;jenkins-hbase4:36613] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-16 21:16:06,908 INFO [RS:2;jenkins-hbase4:36613] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-16 21:16:06,914 INFO [RS:0;jenkins-hbase4:44109] regionserver.Replication(203): jenkins-hbase4.apache.org,44109,1689542165940 started 2023-07-16 21:16:06,914 INFO [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,44109,1689542165940, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:44109, sessionid=0x1016ffd8078001d 2023-07-16 21:16:06,915 DEBUG [RS:0;jenkins-hbase4:44109] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-16 21:16:06,915 DEBUG [RS:0;jenkins-hbase4:44109] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:06,915 DEBUG [RS:0;jenkins-hbase4:44109] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44109,1689542165940' 2023-07-16 21:16:06,915 DEBUG [RS:0;jenkins-hbase4:44109] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-16 21:16:06,915 DEBUG [RS:0;jenkins-hbase4:44109] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-16 21:16:06,915 DEBUG [RS:0;jenkins-hbase4:44109] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-16 21:16:06,915 DEBUG [RS:0;jenkins-hbase4:44109] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-16 21:16:06,916 DEBUG [RS:0;jenkins-hbase4:44109] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:06,916 DEBUG [RS:0;jenkins-hbase4:44109] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44109,1689542165940' 2023-07-16 21:16:06,916 DEBUG [RS:0;jenkins-hbase4:44109] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:16:06,916 DEBUG [RS:0;jenkins-hbase4:44109] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:16:06,916 DEBUG [RS:0;jenkins-hbase4:44109] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-16 21:16:06,916 INFO [RS:0;jenkins-hbase4:44109] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-16 21:16:06,916 INFO [RS:0;jenkins-hbase4:44109] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-16 21:16:06,956 WARN [ReadOnlyZKClient-127.0.0.1:63917@0x184da964] client.ZKConnectionRegistry(168): Meta region is in state OPENING 2023-07-16 21:16:06,957 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33973,1689542165780] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:16:06,958 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55208, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:16:06,959 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40937] ipc.CallRunner(144): callId: 2 service: ClientService methodName: Get size: 88 connection: 172.31.14.131:55208 deadline: 1689542226959, exception=org.apache.hadoop.hbase.NotServingRegionException: hbase:meta,,1 is not online on jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:06,999 INFO [RS:1;jenkins-hbase4:40937] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40937%2C1689542166090, suffix=, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,40937,1689542166090, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs, maxLogs=32 2023-07-16 21:16:07,010 INFO [RS:2;jenkins-hbase4:36613] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36613%2C1689542166251, suffix=, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,36613,1689542166251, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs, maxLogs=32 2023-07-16 21:16:07,015 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:07,017 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-16 21:16:07,019 INFO [RS:0;jenkins-hbase4:44109] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44109%2C1689542165940, suffix=, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,44109,1689542165940, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs, maxLogs=32 2023-07-16 21:16:07,020 INFO [RS-EventLoopGroup-15-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55218, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-16 21:16:07,023 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:16:07,023 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:16:07,023 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:16:07,029 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-16 21:16:07,029 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:16:07,033 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40937%2C1689542166090.meta, suffix=.meta, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,40937,1689542166090, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs, maxLogs=32 2023-07-16 21:16:07,036 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:16:07,036 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:16:07,036 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:16:07,037 INFO [RS:1;jenkins-hbase4:40937] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,40937,1689542166090/jenkins-hbase4.apache.org%2C40937%2C1689542166090.1689542167000 2023-07-16 21:16:07,049 DEBUG [RS:1;jenkins-hbase4:40937] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK], DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK], DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK]] 2023-07-16 21:16:07,050 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:16:07,050 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:16:07,050 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:16:07,051 INFO [RS:2;jenkins-hbase4:36613] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,36613,1689542166251/jenkins-hbase4.apache.org%2C36613%2C1689542166251.1689542167010 2023-07-16 21:16:07,051 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:16:07,051 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:16:07,052 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:16:07,056 DEBUG [RS:2;jenkins-hbase4:36613] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK], DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK], DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK]] 2023-07-16 21:16:07,063 INFO [RS:0;jenkins-hbase4:44109] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,44109,1689542165940/jenkins-hbase4.apache.org%2C44109%2C1689542165940.1689542167020 2023-07-16 21:16:07,066 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,40937,1689542166090/jenkins-hbase4.apache.org%2C40937%2C1689542166090.meta.1689542167033.meta 2023-07-16 21:16:07,066 DEBUG [RS:0;jenkins-hbase4:44109] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK], DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK], DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK]] 2023-07-16 21:16:07,067 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK], DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK], DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK]] 2023-07-16 21:16:07,068 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:16:07,068 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-16 21:16:07,068 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-16 21:16:07,068 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-16 21:16:07,068 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-16 21:16:07,068 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:16:07,069 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-16 21:16:07,069 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-16 21:16:07,071 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-16 21:16:07,072 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info 2023-07-16 21:16:07,072 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info 2023-07-16 21:16:07,073 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-16 21:16:07,083 INFO [StoreFileOpener-info-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c49957a900a44abbf01b64d8b340ddd 2023-07-16 21:16:07,083 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/8c49957a900a44abbf01b64d8b340ddd 2023-07-16 21:16:07,089 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/ca6165e191fc4d699c6c92331ab746d5 2023-07-16 21:16:07,089 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:16:07,089 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-16 21:16:07,090 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/rep_barrier 2023-07-16 21:16:07,090 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/rep_barrier 2023-07-16 21:16:07,091 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-16 21:16:07,107 INFO [StoreFileOpener-rep_barrier-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a387d081595450199b46921f8b69aeb 2023-07-16 21:16:07,107 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/rep_barrier/7a387d081595450199b46921f8b69aeb 2023-07-16 21:16:07,107 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:16:07,107 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-16 21:16:07,108 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table 2023-07-16 21:16:07,109 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table 2023-07-16 21:16:07,109 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-16 21:16:07,121 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table/39ca1cf509684b6ab736d32a3c68a7be 2023-07-16 21:16:07,130 INFO [StoreFileOpener-table-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e6b7923a5b4640e084b40f28aa6202c9 2023-07-16 21:16:07,130 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table/e6b7923a5b4640e084b40f28aa6202c9 2023-07-16 21:16:07,130 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:16:07,131 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740 2023-07-16 21:16:07,132 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740 2023-07-16 21:16:07,135 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-16 21:16:07,136 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-16 21:16:07,138 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=153; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11749310880, jitterRate=0.09423984587192535}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-16 21:16:07,138 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-16 21:16:07,139 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=123, masterSystemTime=1689542167015 2023-07-16 21:16:07,143 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-16 21:16:07,144 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-16 21:16:07,144 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40937,1689542166090, state=OPEN 2023-07-16 21:16:07,146 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-16 21:16:07,146 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-16 21:16:07,148 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=122 2023-07-16 21:16:07,148 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=122, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40937,1689542166090 in 300 msec 2023-07-16 21:16:07,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-07-16 21:16:07,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 470 msec 2023-07-16 21:16:07,282 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33973,1689542165780] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:34991 this server is in the failed servers list 2023-07-16 21:16:07,388 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33973,1689542165780] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:34991 this server is in the failed servers list 2023-07-16 21:16:07,593 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33973,1689542165780] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:34991 this server is in the failed servers list 2023-07-16 21:16:07,898 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33973,1689542165780] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:34991 this server is in the failed servers list 2023-07-16 21:16:08,326 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=3; waited=1653ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-16 21:16:08,403 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33973,1689542165780] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:34991 this server is in the failed servers list 2023-07-16 21:16:09,411 WARN [RS-EventLoopGroup-16-3] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase4.apache.org/172.31.14.131:34991 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:34991 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-16 21:16:09,413 DEBUG [RS-EventLoopGroup-16-3] ipc.FailedServers(52): Added failed server with address jenkins-hbase4.apache.org/172.31.14.131:34991 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:34991 2023-07-16 21:16:09,831 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=3; waited=3157ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3007ms 2023-07-16 21:16:11,183 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=4510ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-16 21:16:11,183 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-16 21:16:11,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=73c0efa65c224047ddc5ff1870c16b5e, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34345,1689542153062, regionLocation=jenkins-hbase4.apache.org,34345,1689542153062, openSeqNum=15 2023-07-16 21:16:11,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=08c6c5dca9905cead995e8dedf4b2e19, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34991,1689542153429, regionLocation=jenkins-hbase4.apache.org,34991,1689542153429, openSeqNum=2 2023-07-16 21:16:11,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=82d35ab01095f590b40a2bc17e2059db, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34991,1689542153429, regionLocation=jenkins-hbase4.apache.org,34991,1689542153429, openSeqNum=71 2023-07-16 21:16:11,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-16 21:16:11,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689542231186 2023-07-16 21:16:11,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689542291186 2023-07-16 21:16:11,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-16 21:16:11,203 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,34991,1689542153429 had 3 regions 2023-07-16 21:16:11,203 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,41419,1689542153256 had 0 regions 2023-07-16 21:16:11,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33973,1689542165780-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:11,203 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,34345,1689542153062 had 1 regions 2023-07-16 21:16:11,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33973,1689542165780-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:11,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33973,1689542165780-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:11,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33973, period=300000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:11,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:11,203 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. is NOT online; state={73c0efa65c224047ddc5ff1870c16b5e state=OPEN, ts=1689542171186, server=jenkins-hbase4.apache.org,34345,1689542153062}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-16 21:16:11,205 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=121, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34991,1689542153429, splitWal=true, meta=true, isMeta: false 2023-07-16 21:16:11,205 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=119, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34345,1689542153062, splitWal=true, meta=false, isMeta: false 2023-07-16 21:16:11,205 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=120, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,41419,1689542153256, splitWal=true, meta=false, isMeta: false 2023-07-16 21:16:11,208 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,34345,1689542153062/hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e., unknown_server=jenkins-hbase4.apache.org,34991,1689542153429/hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19., unknown_server=jenkins-hbase4.apache.org,34991,1689542153429/hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:11,208 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34991,1689542153429-splitting dir is empty, no logs to split. 2023-07-16 21:16:11,208 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,34991,1689542153429 WAL count=0, meta=false 2023-07-16 21:16:11,209 DEBUG [PEWorker-3] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34345,1689542153062-splitting 2023-07-16 21:16:11,210 DEBUG [PEWorker-2] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,41419,1689542153256-splitting 2023-07-16 21:16:11,210 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34345,1689542153062-splitting dir is empty, no logs to split. 2023-07-16 21:16:11,210 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,34345,1689542153062 WAL count=0, meta=false 2023-07-16 21:16:11,211 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,41419,1689542153256-splitting dir is empty, no logs to split. 2023-07-16 21:16:11,211 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,41419,1689542153256 WAL count=0, meta=false 2023-07-16 21:16:11,212 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34991,1689542153429-splitting dir is empty, no logs to split. 2023-07-16 21:16:11,212 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,34991,1689542153429 WAL count=0, meta=false 2023-07-16 21:16:11,212 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34991,1689542153429 WAL splitting is done? wals=0, meta=false 2023-07-16 21:16:11,213 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34345,1689542153062-splitting dir is empty, no logs to split. 2023-07-16 21:16:11,213 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,34345,1689542153062 WAL count=0, meta=false 2023-07-16 21:16:11,213 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34345,1689542153062 WAL splitting is done? wals=0, meta=false 2023-07-16 21:16:11,214 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,41419,1689542153256-splitting dir is empty, no logs to split. 2023-07-16 21:16:11,214 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=124, ppid=121, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:quota, region=08c6c5dca9905cead995e8dedf4b2e19, ASSIGN}, {pid=125, ppid=121, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, ASSIGN}] 2023-07-16 21:16:11,214 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,41419,1689542153256 WAL count=0, meta=false 2023-07-16 21:16:11,214 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,41419,1689542153256 WAL splitting is done? wals=0, meta=false 2023-07-16 21:16:11,215 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=124, ppid=121, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:quota, region=08c6c5dca9905cead995e8dedf4b2e19, ASSIGN 2023-07-16 21:16:11,215 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=125, ppid=121, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, ASSIGN 2023-07-16 21:16:11,216 INFO [PEWorker-3] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,34345,1689542153062 failed, ignore...File hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34345,1689542153062-splitting does not exist. 2023-07-16 21:16:11,216 INFO [PEWorker-2] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,41419,1689542153256 failed, ignore...File hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,41419,1689542153256-splitting does not exist. 2023-07-16 21:16:11,217 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=124, ppid=121, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:quota, region=08c6c5dca9905cead995e8dedf4b2e19, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-16 21:16:11,217 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, ASSIGN}] 2023-07-16 21:16:11,217 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=125, ppid=121, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-16 21:16:11,219 DEBUG [jenkins-hbase4:33973] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-07-16 21:16:11,219 DEBUG [jenkins-hbase4:33973] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-16 21:16:11,219 DEBUG [jenkins-hbase4:33973] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-16 21:16:11,219 DEBUG [jenkins-hbase4:33973] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-16 21:16:11,219 DEBUG [jenkins-hbase4:33973] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-16 21:16:11,219 DEBUG [jenkins-hbase4:33973] balancer.BaseLoadBalancer$Cluster(378): Number of tables=2, number of hosts=1, number of racks=1 2023-07-16 21:16:11,221 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=124 updating hbase:meta row=08c6c5dca9905cead995e8dedf4b2e19, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:11,221 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1689542171221"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542171221"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542171221"}]},"ts":"1689542171221"} 2023-07-16 21:16:11,222 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,41419,1689542153256 after splitting done 2023-07-16 21:16:11,222 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,41419,1689542153256 from processing; numProcessing=2 2023-07-16 21:16:11,222 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=125 updating hbase:meta row=82d35ab01095f590b40a2bc17e2059db, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:11,222 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, ASSIGN 2023-07-16 21:16:11,222 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1689542171222"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542171222"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542171222"}]},"ts":"1689542171222"} 2023-07-16 21:16:11,224 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-16 21:16:11,224 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=124, state=RUNNABLE; OpenRegionProcedure 08c6c5dca9905cead995e8dedf4b2e19, server=jenkins-hbase4.apache.org,44109,1689542165940}] 2023-07-16 21:16:11,225 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=120, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,41419,1689542153256, splitWal=true, meta=false in 4.6180 sec 2023-07-16 21:16:11,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=128, ppid=125, state=RUNNABLE; OpenRegionProcedure 82d35ab01095f590b40a2bc17e2059db, server=jenkins-hbase4.apache.org,36613,1689542166251}] 2023-07-16 21:16:11,374 DEBUG [jenkins-hbase4:33973] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-07-16 21:16:11,374 DEBUG [jenkins-hbase4:33973] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-16 21:16:11,374 DEBUG [jenkins-hbase4:33973] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-16 21:16:11,374 DEBUG [jenkins-hbase4:33973] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-16 21:16:11,374 DEBUG [jenkins-hbase4:33973] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-16 21:16:11,374 DEBUG [jenkins-hbase4:33973] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-16 21:16:11,376 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=73c0efa65c224047ddc5ff1870c16b5e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:11,377 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689542171376"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542171376"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542171376"}]},"ts":"1689542171376"} 2023-07-16 21:16:11,378 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:11,378 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:11,378 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-16 21:16:11,379 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-16 21:16:11,380 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=126, state=RUNNABLE; OpenRegionProcedure 73c0efa65c224047ddc5ff1870c16b5e, server=jenkins-hbase4.apache.org,40937,1689542166090}] 2023-07-16 21:16:11,380 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48880, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-16 21:16:11,380 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43782, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-16 21:16:11,390 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:11,390 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:16:11,390 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 82d35ab01095f590b40a2bc17e2059db, NAME => 'hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:16:11,390 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 08c6c5dca9905cead995e8dedf4b2e19, NAME => 'hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:16:11,390 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-16 21:16:11,390 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. service=MultiRowMutationService 2023-07-16 21:16:11,390 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table quota 08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:16:11,390 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:rsgroup successfully. 2023-07-16 21:16:11,390 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:16:11,390 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table rsgroup 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:11,391 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:16:11,391 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:16:11,391 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:16:11,391 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:11,391 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:11,392 INFO [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family q of region 08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:16:11,392 INFO [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family m of region 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:11,393 DEBUG [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19/q 2023-07-16 21:16:11,393 DEBUG [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19/q 2023-07-16 21:16:11,393 DEBUG [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m 2023-07-16 21:16:11,393 DEBUG [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m 2023-07-16 21:16:11,394 INFO [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 08c6c5dca9905cead995e8dedf4b2e19 columnFamilyName q 2023-07-16 21:16:11,394 INFO [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 82d35ab01095f590b40a2bc17e2059db columnFamilyName m 2023-07-16 21:16:11,394 INFO [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] regionserver.HStore(310): Store=08c6c5dca9905cead995e8dedf4b2e19/q, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:16:11,394 INFO [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family u of region 08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:16:11,395 DEBUG [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19/u 2023-07-16 21:16:11,395 DEBUG [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19/u 2023-07-16 21:16:11,395 INFO [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 08c6c5dca9905cead995e8dedf4b2e19 columnFamilyName u 2023-07-16 21:16:11,396 INFO [StoreOpener-08c6c5dca9905cead995e8dedf4b2e19-1] regionserver.HStore(310): Store=08c6c5dca9905cead995e8dedf4b2e19/u, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:16:11,397 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:16:11,398 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:16:11,403 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:quota descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-16 21:16:11,405 DEBUG [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/23d165da51744f1692de8df99d4f87ef 2023-07-16 21:16:11,405 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:16:11,406 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 08c6c5dca9905cead995e8dedf4b2e19; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11258443520, jitterRate=0.04852426052093506}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-16 21:16:11,406 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 08c6c5dca9905cead995e8dedf4b2e19: 2023-07-16 21:16:11,407 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19., pid=127, masterSystemTime=1689542171378 2023-07-16 21:16:11,412 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:16:11,412 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:16:11,413 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=124 updating hbase:meta row=08c6c5dca9905cead995e8dedf4b2e19, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:11,413 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1689542171412"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542171412"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542171412"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542171412"}]},"ts":"1689542171412"} 2023-07-16 21:16:11,414 INFO [StoreFileOpener-m-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d185255a3a8e4435a58d7c4f27f39d36 2023-07-16 21:16:11,414 DEBUG [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/d185255a3a8e4435a58d7c4f27f39d36 2023-07-16 21:16:11,414 INFO [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] regionserver.HStore(310): Store=82d35ab01095f590b40a2bc17e2059db/m, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:16:11,415 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:11,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=124 2023-07-16 21:16:11,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=124, state=SUCCESS; OpenRegionProcedure 08c6c5dca9905cead995e8dedf4b2e19, server=jenkins-hbase4.apache.org,44109,1689542165940 in 190 msec 2023-07-16 21:16:11,417 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:11,419 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=124, ppid=121, state=SUCCESS; TransitRegionStateProcedure table=hbase:quota, region=08c6c5dca9905cead995e8dedf4b2e19, ASSIGN in 203 msec 2023-07-16 21:16:11,421 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:11,422 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 82d35ab01095f590b40a2bc17e2059db; next sequenceid=78; org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy@5d5191de, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:16:11,422 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 82d35ab01095f590b40a2bc17e2059db: 2023-07-16 21:16:11,423 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db., pid=128, masterSystemTime=1689542171378 2023-07-16 21:16:11,426 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:11,427 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:11,427 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=125 updating hbase:meta row=82d35ab01095f590b40a2bc17e2059db, regionState=OPEN, openSeqNum=78, regionLocation=jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:11,427 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1689542171427"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542171427"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542171427"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542171427"}]},"ts":"1689542171427"} 2023-07-16 21:16:11,431 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=128, resume processing ppid=125 2023-07-16 21:16:11,431 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=128, ppid=125, state=SUCCESS; OpenRegionProcedure 82d35ab01095f590b40a2bc17e2059db, server=jenkins-hbase4.apache.org,36613,1689542166251 in 204 msec 2023-07-16 21:16:11,433 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=125, resume processing ppid=121 2023-07-16 21:16:11,433 INFO [PEWorker-4] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,34991,1689542153429 after splitting done 2023-07-16 21:16:11,433 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=125, ppid=121, state=SUCCESS; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, ASSIGN in 217 msec 2023-07-16 21:16:11,433 DEBUG [PEWorker-4] master.DeadServer(114): Removed jenkins-hbase4.apache.org,34991,1689542153429 from processing; numProcessing=1 2023-07-16 21:16:11,435 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=121, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34991,1689542153429, splitWal=true, meta=true in 4.8240 sec 2023-07-16 21:16:11,435 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33973,1689542165780] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:16:11,436 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43792, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:16:11,438 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker(840): RSGroup table=hbase:rsgroup is online, refreshing cached information 2023-07-16 21:16:11,438 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl(534): Refreshing in Online mode. 2023-07-16 21:16:11,449 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:11,450 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:16:11,450 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:16:11,457 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rsgroup 2023-07-16 21:16:11,457 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker(823): GroupBasedLoadBalancer is now online 2023-07-16 21:16:11,536 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:11,536 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 73c0efa65c224047ddc5ff1870c16b5e, NAME => 'hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:16:11,536 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:11,536 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:16:11,536 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:11,536 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:11,538 INFO [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:11,539 DEBUG [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/info 2023-07-16 21:16:11,539 DEBUG [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/info 2023-07-16 21:16:11,539 INFO [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 73c0efa65c224047ddc5ff1870c16b5e columnFamilyName info 2023-07-16 21:16:11,546 INFO [StoreFileOpener-info-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7aca6aa2367450cb461f1ac0a602f75 2023-07-16 21:16:11,546 DEBUG [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/info/e7aca6aa2367450cb461f1ac0a602f75 2023-07-16 21:16:11,546 INFO [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] regionserver.HStore(310): Store=73c0efa65c224047ddc5ff1870c16b5e/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:16:11,547 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:11,549 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:11,552 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:11,555 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 73c0efa65c224047ddc5ff1870c16b5e; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=9457713760, jitterRate=-0.11918176710605621}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:16:11,555 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 73c0efa65c224047ddc5ff1870c16b5e: 2023-07-16 21:16:11,556 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e., pid=129, masterSystemTime=1689542171532 2023-07-16 21:16:11,557 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:11,557 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:11,558 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=73c0efa65c224047ddc5ff1870c16b5e, regionState=OPEN, openSeqNum=18, regionLocation=jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:11,558 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689542171558"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542171558"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542171558"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542171558"}]},"ts":"1689542171558"} 2023-07-16 21:16:11,561 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=126 2023-07-16 21:16:11,562 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=126, state=SUCCESS; OpenRegionProcedure 73c0efa65c224047ddc5ff1870c16b5e, server=jenkins-hbase4.apache.org,40937,1689542166090 in 180 msec 2023-07-16 21:16:11,563 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=119 2023-07-16 21:16:11,563 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,34345,1689542153062 after splitting done 2023-07-16 21:16:11,563 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, ASSIGN in 344 msec 2023-07-16 21:16:11,563 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,34345,1689542153062 from processing; numProcessing=0 2023-07-16 21:16:11,564 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34345,1689542153062, splitWal=true, meta=false in 4.9640 sec 2023-07-16 21:16:12,204 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/namespace 2023-07-16 21:16:12,222 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-16 21:16:12,224 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-07-16 21:16:12,224 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.789sec 2023-07-16 21:16:12,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-16 21:16:12,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-16 21:16:12,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-16 21:16:12,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33973,1689542165780-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-16 21:16:12,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33973,1689542165780-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-16 21:16:12,227 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-16 21:16:12,323 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(139): Connect 0x2ac60d16 to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:16:12,334 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2c814f8b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:16:12,336 DEBUG [hconnection-0x158b00df-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:16:12,338 INFO [RS-EventLoopGroup-15-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36532, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:16:12,344 INFO [Listener at localhost/43605] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-16 21:16:12,344 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2ac60d16 to 127.0.0.1:63917 2023-07-16 21:16:12,345 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:12,346 INFO [Listener at localhost/43605] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:33973 after: jenkins-hbase4.apache.org:33973 2023-07-16 21:16:12,346 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(139): Connect 0x06aa2997 to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:16:12,350 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4bea4c65, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:16:12,351 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:16:12,351 DEBUG [Listener at localhost/43605] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-16 21:16:12,353 INFO [RS-EventLoopGroup-13-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45844, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-16 21:16:12,355 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-16 21:16:12,355 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-16 21:16:12,356 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(139): Connect 0x5f24fc8d to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:16:12,363 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@71435113, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:16:12,363 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=VerifyingRSGroupAdminClient connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:16:12,365 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): VerifyingRSGroupAdminClient0x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:16:12,367 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): VerifyingRSGroupAdminClient-0x1016ffd80780027 connected 2023-07-16 21:16:12,370 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:16:12,371 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:16:12,372 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:16:12,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:16:12,373 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:16:12,373 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:16:12,373 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:16:12,374 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:16:12,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:12,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:16:12,379 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:16:12,383 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 1 2023-07-16 21:16:12,395 INFO [Listener at localhost/43605] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:16:12,395 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:12,395 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:12,395 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:16:12,395 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:12,395 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:16:12,395 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:16:12,396 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34749 2023-07-16 21:16:12,396 INFO [Listener at localhost/43605] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-16 21:16:12,400 DEBUG [Listener at localhost/43605] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-16 21:16:12,401 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:16:12,402 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:16:12,403 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34749 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:16:12,409 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:347490x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:16:12,410 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(162): regionserver:347490x0, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-16 21:16:12,411 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34749-0x1016ffd80780028 connected 2023-07-16 21:16:12,412 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(162): regionserver:34749-0x1016ffd80780028, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/running 2023-07-16 21:16:12,413 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:34749-0x1016ffd80780028, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:16:12,414 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34749 2023-07-16 21:16:12,422 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34749 2023-07-16 21:16:12,423 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34749 2023-07-16 21:16:12,425 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34749 2023-07-16 21:16:12,426 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34749 2023-07-16 21:16:12,427 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:16:12,427 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:16:12,427 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:16:12,428 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-16 21:16:12,428 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:16:12,428 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:16:12,428 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:16:12,429 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 41033 2023-07-16 21:16:12,429 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:16:12,430 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:12,430 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@6e5a7c27{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:16:12,430 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:12,431 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@2cc42138{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:16:12,554 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:16:12,554 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:16:12,555 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:16:12,555 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 600000ms 2023-07-16 21:16:12,556 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:12,556 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@7cfd6419{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-41033-hbase-server-2_4_18-SNAPSHOT_jar-_-any-417539345003808387/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:16:12,558 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@2028dbf3{HTTP/1.1, (http/1.1)}{0.0.0.0:41033} 2023-07-16 21:16:12,558 INFO [Listener at localhost/43605] server.Server(415): Started @46422ms 2023-07-16 21:16:12,561 INFO [RS:3;jenkins-hbase4:34749] regionserver.HRegionServer(951): ClusterId : bc8d5a8a-d305-489f-bc1d-b95b98296a6c 2023-07-16 21:16:12,562 DEBUG [RS:3;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-16 21:16:12,564 DEBUG [RS:3;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-16 21:16:12,564 DEBUG [RS:3;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-16 21:16:12,565 DEBUG [RS:3;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-16 21:16:12,568 DEBUG [RS:3;jenkins-hbase4:34749] zookeeper.ReadOnlyZKClient(139): Connect 0x153625b0 to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:16:12,572 DEBUG [RS:3;jenkins-hbase4:34749] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1bf1d966, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:16:12,572 DEBUG [RS:3;jenkins-hbase4:34749] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1fce4ec0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:16:12,576 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-16 21:16:12,585 DEBUG [RS:3;jenkins-hbase4:34749] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase4:34749 2023-07-16 21:16:12,585 INFO [RS:3;jenkins-hbase4:34749] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-16 21:16:12,585 INFO [RS:3;jenkins-hbase4:34749] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-16 21:16:12,585 DEBUG [RS:3;jenkins-hbase4:34749] regionserver.HRegionServer(1022): About to register with Master. 2023-07-16 21:16:12,586 INFO [RS:3;jenkins-hbase4:34749] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33973,1689542165780 with isa=jenkins-hbase4.apache.org/172.31.14.131:34749, startcode=1689542172394 2023-07-16 21:16:12,586 DEBUG [RS:3;jenkins-hbase4:34749] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-16 21:16:12,589 INFO [RS-EventLoopGroup-13-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33481, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.11 (auth:SIMPLE), service=RegionServerStatusService 2023-07-16 21:16:12,590 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33973] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34749,1689542172394 2023-07-16 21:16:12,590 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:16:12,590 DEBUG [RS:3;jenkins-hbase4:34749] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb 2023-07-16 21:16:12,590 DEBUG [RS:3;jenkins-hbase4:34749] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35261 2023-07-16 21:16:12,590 DEBUG [RS:3;jenkins-hbase4:34749] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=38591 2023-07-16 21:16:12,593 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:12,593 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:12,593 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:12,593 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:12,593 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:12,594 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34749,1689542172394] 2023-07-16 21:16:12,594 DEBUG [RS:3;jenkins-hbase4:34749] zookeeper.ZKUtil(162): regionserver:34749-0x1016ffd80780028, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34749,1689542172394 2023-07-16 21:16:12,594 WARN [RS:3;jenkins-hbase4:34749] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:16:12,594 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34749,1689542172394 2023-07-16 21:16:12,594 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 2 2023-07-16 21:16:12,594 INFO [RS:3;jenkins-hbase4:34749] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:16:12,594 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34749,1689542172394 2023-07-16 21:16:12,595 DEBUG [RS:3;jenkins-hbase4:34749] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34749,1689542172394 2023-07-16 21:16:12,595 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:12,596 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 4 2023-07-16 21:16:12,596 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:12,596 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:12,596 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34749,1689542172394 2023-07-16 21:16:12,596 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:12,596 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:12,596 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:12,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:12,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:12,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:12,605 DEBUG [RS:3;jenkins-hbase4:34749] zookeeper.ZKUtil(162): regionserver:34749-0x1016ffd80780028, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34749,1689542172394 2023-07-16 21:16:12,606 DEBUG [RS:3;jenkins-hbase4:34749] zookeeper.ZKUtil(162): regionserver:34749-0x1016ffd80780028, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:12,606 DEBUG [RS:3;jenkins-hbase4:34749] zookeeper.ZKUtil(162): regionserver:34749-0x1016ffd80780028, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:12,606 DEBUG [RS:3;jenkins-hbase4:34749] zookeeper.ZKUtil(162): regionserver:34749-0x1016ffd80780028, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:12,607 DEBUG [RS:3;jenkins-hbase4:34749] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-16 21:16:12,607 INFO [RS:3;jenkins-hbase4:34749] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-16 21:16:12,642 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.quotas.MasterQuotasObserver 2023-07-16 21:16:12,642 INFO [RS:3;jenkins-hbase4:34749] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-16 21:16:12,643 INFO [RS:3;jenkins-hbase4:34749] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-16 21:16:12,643 INFO [RS:3;jenkins-hbase4:34749] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:12,643 INFO [RS:3;jenkins-hbase4:34749] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-16 21:16:12,646 INFO [RS:3;jenkins-hbase4:34749] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:12,646 DEBUG [RS:3;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:12,646 DEBUG [RS:3;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:12,646 DEBUG [RS:3;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:12,646 DEBUG [RS:3;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:12,646 DEBUG [RS:3;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:12,646 DEBUG [RS:3;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:16:12,646 DEBUG [RS:3;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:12,646 DEBUG [RS:3;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:12,646 DEBUG [RS:3;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:12,646 DEBUG [RS:3;jenkins-hbase4:34749] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:12,648 INFO [RS:3;jenkins-hbase4:34749] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:12,648 INFO [RS:3;jenkins-hbase4:34749] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:12,648 INFO [RS:3;jenkins-hbase4:34749] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:12,665 INFO [RS:3;jenkins-hbase4:34749] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-16 21:16:12,666 INFO [RS:3;jenkins-hbase4:34749] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34749,1689542172394-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:12,682 INFO [RS:3;jenkins-hbase4:34749] regionserver.Replication(203): jenkins-hbase4.apache.org,34749,1689542172394 started 2023-07-16 21:16:12,682 INFO [RS:3;jenkins-hbase4:34749] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34749,1689542172394, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34749, sessionid=0x1016ffd80780028 2023-07-16 21:16:12,682 DEBUG [RS:3;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-16 21:16:12,682 DEBUG [RS:3;jenkins-hbase4:34749] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34749,1689542172394 2023-07-16 21:16:12,682 DEBUG [RS:3;jenkins-hbase4:34749] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34749,1689542172394' 2023-07-16 21:16:12,682 DEBUG [RS:3;jenkins-hbase4:34749] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-16 21:16:12,683 DEBUG [RS:3;jenkins-hbase4:34749] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-16 21:16:12,683 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:16:12,683 DEBUG [RS:3;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-16 21:16:12,684 DEBUG [RS:3;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-16 21:16:12,684 DEBUG [RS:3;jenkins-hbase4:34749] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34749,1689542172394 2023-07-16 21:16:12,684 DEBUG [RS:3;jenkins-hbase4:34749] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34749,1689542172394' 2023-07-16 21:16:12,684 DEBUG [RS:3;jenkins-hbase4:34749] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:16:12,684 DEBUG [RS:3;jenkins-hbase4:34749] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:16:12,684 DEBUG [RS:3;jenkins-hbase4:34749] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-16 21:16:12,684 INFO [RS:3;jenkins-hbase4:34749] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-16 21:16:12,684 INFO [RS:3;jenkins-hbase4:34749] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-16 21:16:12,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:12,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:16:12,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:16:12,689 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:16:12,691 DEBUG [hconnection-0x2672383d-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:16:12,694 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36536, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:16:12,701 DEBUG [hconnection-0x2672383d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:16:12,703 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43804, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:16:12,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:16:12,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:16:12,709 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:33973] to rsgroup master 2023-07-16 21:16:12,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:33973 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:16:12,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] ipc.CallRunner(144): callId: 25 service: MasterService methodName: ExecMasterService size: 118 connection: 172.31.14.131:45844 deadline: 1689543372709, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:33973 is either offline or it does not exist. 2023-07-16 21:16:12,709 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:33973 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:33973 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:16:12,711 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:16:12,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:16:12,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:16:12,712 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34749, jenkins-hbase4.apache.org:36613, jenkins-hbase4.apache.org:40937, jenkins-hbase4.apache.org:44109], Tables:[hbase:meta, hbase:quota, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:16:12,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:16:12,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:16:12,765 INFO [Listener at localhost/43605] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testRSGroupsWithHBaseQuota Thread=565 (was 514) Potentially hanging thread: AsyncFSWAL-0-hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb-prefix:jenkins-hbase4.apache.org,40937,1689542166090.meta sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.6@localhost:35261 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp664559603-1621 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/234338121.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741891_1067, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741888_1064, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1665386889-1594 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp2088989524-1656 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (409621679) connection to localhost/127.0.0.1:35261 from jenkins.hfs.8 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: qtp1665386889-1592 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33973 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=36613 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS-EventLoopGroup-15-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36613 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp664559603-1622-acceptor-0@2c0466bd-ServerConnector@505610f7{HTTP/1.1, (http/1.1)}{0.0.0.0:45049} sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.hbase.thirdparty.org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) org.apache.hbase.thirdparty.org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:704) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=44109 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp1078873375-1915 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1078873375-1916 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-17-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:302) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:366) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp401889684-1694 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/234338121.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:3;jenkins-hbase4:34749 java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1092) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:175) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:123) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:159) java.security.AccessController.doPrivileged(Native Method) javax.security.auth.Subject.doAs(Subject.java:360) org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1873) org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:319) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:156) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741889_1065, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-348021530_17 at /127.0.0.1:55676 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741889_1065] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_128997302_17 at /127.0.0.1:41710 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741890_1066] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.8@localhost:35261 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x3176a954-SendThread(127.0.0.1:63917) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: qtp1665386889-1595 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: AsyncFSWAL-0-hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb-prefix:jenkins-hbase4.apache.org,44109,1689542165940 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.9@localhost:35261 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp2088989524-1658 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-348021530_17 at /127.0.0.1:41734 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741892_1068] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: jenkins-hbase4:40937Replication Statistics #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x5f24fc8d-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44109 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34749 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp401889684-1698 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp664559603-1623 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=36613 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=33973 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x06aa2997-SendThread(127.0.0.1:63917) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: IPC Client (409621679) connection to localhost/127.0.0.1:35261 from jenkins.hfs.10 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: Session-HouseKeeper-7c49def6-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3c27e1db-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp2051740302-1681 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/234338121.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3c27e1db-metaLookup-shared--pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1078873375-1913 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/234338121.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: jenkins-hbase4:44109Replication Statistics #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-348021530_17 at /127.0.0.1:41700 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741889_1065] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Session-HouseKeeper-3a6788fa-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp2051740302-1682-acceptor-0@29062cd9-ServerConnector@21d758b1{HTTP/1.1, (http/1.1)}{0.0.0.0:37073} sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.hbase.thirdparty.org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) org.apache.hbase.thirdparty.org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:704) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x792f238d-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x3716accb sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$93/298992337.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40937 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp1078873375-1919 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-14 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x3176a954 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$93/298992337.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x7ba085ac sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$93/298992337.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40937 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-11 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp2051740302-1687 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1300024448_17 at /127.0.0.1:41720 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741891_1067] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp2088989524-1655 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1665386889-1597 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741890_1066, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:3;jenkins-hbase4:34749-longCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp401889684-1695 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/234338121.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33729,1689542152890 java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread.run(RSGroupInfoManagerImpl.java:797) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34749 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp664559603-1627 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33973 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741890_1066, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x184da964 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$93/298992337.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x792f238d-SendThread(127.0.0.1:63917) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44109 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741891_1067, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:35261 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (409621679) connection to localhost/127.0.0.1:35261 from jenkins.hfs.9 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1300024448_17 at /127.0.0.1:47162 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741891_1067] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp664559603-1625 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44109 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741888_1064, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp2051740302-1686 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1613257394_17 at /127.0.0.1:59340 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=44109 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: IPC Client (409621679) connection to localhost/127.0.0.1:35261 from jenkins.hfs.6 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-16-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:302) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:366) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40937 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp401889684-1697 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:302) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:366) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp664559603-1628 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1078873375-1920 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=34749 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: hconnection-0x3c27e1db-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=40937 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741888_1064, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36613 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: jenkins-hbase4:36613Replication Statistics #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-15 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34749 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS-EventLoopGroup-13-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x153625b0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$93/298992337.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34749 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x5f24fc8d-SendThread(127.0.0.1:63917) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: hconnection-0x3c27e1db-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp2088989524-1654 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741892_1068, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: jenkins-hbase4:34749Replication Statistics #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2672383d-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=36613 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x153625b0-SendThread(127.0.0.1:63917) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: qtp1665386889-1590 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/234338121.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-301091760_17 at /127.0.0.1:47148 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741888_1064] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp2088989524-1652-acceptor-0@5986c2ae-ServerConnector@4ede1e50{HTTP/1.1, (http/1.1)}{0.0.0.0:43323} sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.hbase.thirdparty.org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) org.apache.hbase.thirdparty.org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:704) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3c27e1db-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33973 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_899635112_17 at /127.0.0.1:59328 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:44109-longCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=34749 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: IPC Client (409621679) connection to localhost/127.0.0.1:35261 from jenkins.hfs.5 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1613257394_17 at /127.0.0.1:35494 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp2051740302-1688 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:36613-longCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=40937 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp1665386889-1591-acceptor-0@663089b6-ServerConnector@14c4c302{HTTP/1.1, (http/1.1)}{0.0.0.0:38591} sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.hbase.thirdparty.org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) org.apache.hbase.thirdparty.org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:704) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp2051740302-1685 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x06aa2997-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: qtp401889684-1699 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2672383d-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1300024448_17 at /127.0.0.1:55702 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741891_1067] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x184da964-SendThread(127.0.0.1:63917) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689542166670 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:267) org.apache.hadoop.hbase.master.cleaner.HFileCleaner$1.run(HFileCleaner.java:236) Potentially hanging thread: RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=44109 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=44109 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=36613 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS-EventLoopGroup-9-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp2088989524-1653 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x153625b0-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: AsyncFSWAL-0-hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb-prefix:jenkins-hbase4.apache.org,36613,1689542166251 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x3716accb-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: Session-HouseKeeper-74c18fcf-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x5f24fc8d sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$93/298992337.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (409621679) connection to localhost/127.0.0.1:35261 from jenkins.hfs.11 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-348021530_17 at /127.0.0.1:47154 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741889_1065] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp664559603-1626 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36613 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: hconnection-0x3c27e1db-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-12 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-301091760_17 at /127.0.0.1:41684 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741888_1064] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-348021530_17 at /127.0.0.1:47172 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741892_1068] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=33973 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=40937 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-10 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x3716accb-SendThread(127.0.0.1:63917) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: LeaseRenewer:jenkins.hfs.10@localhost:35261 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1078873375-1914-acceptor-0@561f31ec-ServerConnector@2028dbf3{HTTP/1.1, (http/1.1)}{0.0.0.0:41033} sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.hbase.thirdparty.org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) org.apache.hbase.thirdparty.org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:704) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x3176a954-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=40937 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp1078873375-1917 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741890_1066, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-301091760_17 at /127.0.0.1:55660 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741888_1064] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Session-HouseKeeper-61080f19-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: AsyncFSWAL-0-hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb-prefix:jenkins-hbase4.apache.org,40937,1689542166090 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40937 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33973 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: IPC Client (409621679) connection to localhost/127.0.0.1:35261 from jenkins.hfs.7 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.7@localhost:35261 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:36613 java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1092) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:175) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:123) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:159) java.security.AccessController.doPrivileged(Native Method) javax.security.auth.Subject.doAs(Subject.java:360) org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1873) org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:319) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:156) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34749 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_128997302_17 at /127.0.0.1:55686 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741890_1066] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp401889684-1692 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/234338121.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-13 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:44109 java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1092) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:175) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:123) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:159) java.security.AccessController.doPrivileged(Native Method) javax.security.auth.Subject.doAs(Subject.java:360) org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1873) org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:319) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:156) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:40937 java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1092) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:175) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:123) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:159) java.security.AccessController.doPrivileged(Native Method) javax.security.auth.Subject.doAs(Subject.java:360) org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1873) org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:319) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:156) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741889_1065, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36613 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS:1;jenkins-hbase4:40937-longCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36613 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=34749 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: M:0;jenkins-hbase4:33973 java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) org.apache.hadoop.hbase.master.HMaster.waitForMasterActive(HMaster.java:634) org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:957) org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:904) org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1006) org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:541) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44109 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741891_1067, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x06aa2997 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$93/298992337.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp401889684-1696-acceptor-0@4f7f3fe1-ServerConnector@1933a51a{HTTP/1.1, (http/1.1)}{0.0.0.0:38513} sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.hbase.thirdparty.org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) org.apache.hbase.thirdparty.org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:704) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x184da964-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: qtp2088989524-1657 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=33973 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: AsyncFSWAL-0-hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData-prefix:jenkins-hbase4.apache.org,33973,1689542165780 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-348021530_17 at /127.0.0.1:55714 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741892_1068] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:302) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:366) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x7ba085ac-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741889_1065, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34749 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741892_1068, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780 java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread.run(RSGroupInfoManagerImpl.java:797) Potentially hanging thread: hconnection-0x3c27e1db-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1078873375-1918 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: jenkins-hbase4:33973 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.master.assignment.AssignmentManager.waitOnAssignQueue(AssignmentManager.java:2102) org.apache.hadoop.hbase.master.assignment.AssignmentManager.processAssignQueue(AssignmentManager.java:2124) org.apache.hadoop.hbase.master.assignment.AssignmentManager.access$600(AssignmentManager.java:104) org.apache.hadoop.hbase.master.assignment.AssignmentManager$1.run(AssignmentManager.java:2064) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44109 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS-EventLoopGroup-17-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1938682392-172.31.14.131-1689542128252:blk_1073741892_1068, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33973 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: Session-HouseKeeper-37fb7f53-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40937 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1613257394_17 at /127.0.0.1:59342 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1665386889-1593 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36613 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp401889684-1693 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/234338121.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3c27e1db-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=34749 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS-EventLoopGroup-14-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689542166671 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:267) org.apache.hadoop.hbase.master.cleaner.HFileCleaner$2.run(HFileCleaner.java:251) Potentially hanging thread: qtp1665386889-1596 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp664559603-1624 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_128997302_17 at /127.0.0.1:47160 [Receiving block BP-1938682392-172.31.14.131-1689542128252:blk_1073741890_1066] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44109 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40937 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS-EventLoopGroup-13-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33973 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_899635112_17 at /127.0.0.1:59322 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp2051740302-1683 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-16 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp2088989524-1651 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/234338121.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x7ba085ac-SendThread(127.0.0.1:63917) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: qtp2051740302-1684 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:63917@0x792f238d sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$93/298992337.run(Unknown Source) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=843 (was 769) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=300 (was 311), ProcessCount=170 (was 170), AvailableMemoryMB=5236 (was 4879) - AvailableMemoryMB LEAK? - 2023-07-16 21:16:12,768 WARN [Listener at localhost/43605] hbase.ResourceChecker(130): Thread=565 is superior to 500 2023-07-16 21:16:12,786 INFO [RS:3;jenkins-hbase4:34749] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34749%2C1689542172394, suffix=, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34749,1689542172394, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs, maxLogs=32 2023-07-16 21:16:12,789 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:rsgroup' 2023-07-16 21:16:12,791 INFO [Listener at localhost/43605] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testClearDeadServers Thread=564, OpenFileDescriptor=843, MaxFileDescriptor=60000, SystemLoadAverage=300, ProcessCount=170, AvailableMemoryMB=5234 2023-07-16 21:16:12,791 WARN [Listener at localhost/43605] hbase.ResourceChecker(130): Thread=564 is superior to 500 2023-07-16 21:16:12,791 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(132): testClearDeadServers 2023-07-16 21:16:12,795 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:16:12,796 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:16:12,796 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:16:12,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:16:12,797 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:16:12,797 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:16:12,797 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:16:12,798 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:16:12,799 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-16 21:16:12,800 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-16 21:16:12,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:12,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:16:12,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:16:12,816 DEBUG [RS-EventLoopGroup-17-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:16:12,816 DEBUG [RS-EventLoopGroup-17-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:16:12,816 DEBUG [RS-EventLoopGroup-17-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:16:12,816 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:quota' 2023-07-16 21:16:12,818 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-16 21:16:12,819 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:16:12,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:12,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:16:12,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:16:12,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:16:12,831 INFO [RS:3;jenkins-hbase4:34749] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34749,1689542172394/jenkins-hbase4.apache.org%2C34749%2C1689542172394.1689542172787 2023-07-16 21:16:12,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:16:12,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:16:12,834 DEBUG [RS:3;jenkins-hbase4:34749] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK], DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK], DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK]] 2023-07-16 21:16:12,836 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:33973] to rsgroup master 2023-07-16 21:16:12,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:33973 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:16:12,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] ipc.CallRunner(144): callId: 53 service: MasterService methodName: ExecMasterService size: 118 connection: 172.31.14.131:45844 deadline: 1689543372836, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:33973 is either offline or it does not exist. 2023-07-16 21:16:12,837 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:33973 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:33973 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:16:12,839 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:16:12,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:16:12,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:16:12,839 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34749, jenkins-hbase4.apache.org:36613, jenkins-hbase4.apache.org:40937, jenkins-hbase4.apache.org:44109], Tables:[hbase:meta, hbase:quota, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:16:12,840 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:16:12,840 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:16:12,840 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBasics(214): testClearDeadServers 2023-07-16 21:16:12,841 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:16:12,841 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:16:12,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup Group_testClearDeadServers_1449045704 2023-07-16 21:16:12,844 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/Group_testClearDeadServers_1449045704 2023-07-16 21:16:12,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:12,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:16:12,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:16:12,848 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:16:12,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:16:12,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:16:12,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:36613, jenkins-hbase4.apache.org:40937, jenkins-hbase4.apache.org:34749] to rsgroup Group_testClearDeadServers_1449045704 2023-07-16 21:16:12,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:12,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/Group_testClearDeadServers_1449045704 2023-07-16 21:16:12,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:16:12,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:16:12,860 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminServer(238): Moving server region 82d35ab01095f590b40a2bc17e2059db, which do not belong to RSGroup Group_testClearDeadServers_1449045704 2023-07-16 21:16:12,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] procedure2.ProcedureExecutor(1029): Stored pid=130, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, REOPEN/MOVE 2023-07-16 21:16:12,863 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=130, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, REOPEN/MOVE 2023-07-16 21:16:12,864 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=130 updating hbase:meta row=82d35ab01095f590b40a2bc17e2059db, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:12,864 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1689542172864"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542172864"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542172864"}]},"ts":"1689542172864"} 2023-07-16 21:16:12,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminServer(238): Moving server region 73c0efa65c224047ddc5ff1870c16b5e, which do not belong to RSGroup Group_testClearDeadServers_1449045704 2023-07-16 21:16:12,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] procedure2.ProcedureExecutor(1029): Stored pid=131, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, REOPEN/MOVE 2023-07-16 21:16:12,865 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminServer(238): Moving server region 1588230740, which do not belong to RSGroup Group_testClearDeadServers_1449045704 2023-07-16 21:16:12,865 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, REOPEN/MOVE 2023-07-16 21:16:12,865 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=130, state=RUNNABLE; CloseRegionProcedure 82d35ab01095f590b40a2bc17e2059db, server=jenkins-hbase4.apache.org,36613,1689542166251}] 2023-07-16 21:16:12,866 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=73c0efa65c224047ddc5ff1870c16b5e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:12,866 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689542172866"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542172866"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542172866"}]},"ts":"1689542172866"} 2023-07-16 21:16:12,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] procedure2.ProcedureExecutor(1029): Stored pid=132, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, REOPEN/MOVE 2023-07-16 21:16:12,867 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, REOPEN/MOVE 2023-07-16 21:16:12,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminServer(286): Moving 3 region(s) to group default, current retry=0 2023-07-16 21:16:12,869 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40937,1689542166090, state=CLOSING 2023-07-16 21:16:12,869 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; CloseRegionProcedure 73c0efa65c224047ddc5ff1870c16b5e, server=jenkins-hbase4.apache.org,40937,1689542166090}] 2023-07-16 21:16:12,870 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-16 21:16:12,871 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-16 21:16:12,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=135, ppid=132, state=RUNNABLE; CloseRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40937,1689542166090}] 2023-07-16 21:16:13,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:13,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 82d35ab01095f590b40a2bc17e2059db, disabling compactions & flushes 2023-07-16 21:16:13,022 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:13,022 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:13,022 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. after waiting 0 ms 2023-07-16 21:16:13,022 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:13,022 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 82d35ab01095f590b40a2bc17e2059db 1/1 column families, dataSize=2.22 KB heapSize=3.72 KB 2023-07-16 21:16:13,023 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 1588230740 2023-07-16 21:16:13,025 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-16 21:16:13,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:13,025 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-16 21:16:13,025 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-16 21:16:13,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 73c0efa65c224047ddc5ff1870c16b5e, disabling compactions & flushes 2023-07-16 21:16:13,026 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-16 21:16:13,026 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:13,026 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-16 21:16:13,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:13,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. after waiting 0 ms 2023-07-16 21:16:13,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:13,026 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.47 KB heapSize=6.43 KB 2023-07-16 21:16:13,046 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/recovered.edits/20.seqid, newMaxSeqId=20, maxSeqId=17 2023-07-16 21:16:13,048 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:13,048 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 73c0efa65c224047ddc5ff1870c16b5e: 2023-07-16 21:16:13,048 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3513): Adding 73c0efa65c224047ddc5ff1870c16b5e move to jenkins-hbase4.apache.org,44109,1689542165940 record at close sequenceid=18 2023-07-16 21:16:13,052 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:13,055 DEBUG [PEWorker-5] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=134, ppid=131, state=RUNNABLE; CloseRegionProcedure 73c0efa65c224047ddc5ff1870c16b5e, server=jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:13,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.22 KB at sequenceid=88 (bloomFilter=true), to=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/.tmp/m/95305049645a40afbe27bf61c4b28d28 2023-07-16 21:16:13,491 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.47 KB at sequenceid=163 (bloomFilter=false), to=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp/info/fb397dc30b5940959a1c1af2e67ba90f 2023-07-16 21:16:13,497 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 95305049645a40afbe27bf61c4b28d28 2023-07-16 21:16:13,497 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp/info/fb397dc30b5940959a1c1af2e67ba90f as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/fb397dc30b5940959a1c1af2e67ba90f 2023-07-16 21:16:13,498 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/.tmp/m/95305049645a40afbe27bf61c4b28d28 as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/95305049645a40afbe27bf61c4b28d28 2023-07-16 21:16:13,503 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 95305049645a40afbe27bf61c4b28d28 2023-07-16 21:16:13,504 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/95305049645a40afbe27bf61c4b28d28, entries=5, sequenceid=88, filesize=5.3 K 2023-07-16 21:16:13,504 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/fb397dc30b5940959a1c1af2e67ba90f, entries=30, sequenceid=163, filesize=8.2 K 2023-07-16 21:16:13,505 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.22 KB/2276, heapSize ~3.70 KB/3792, currentSize=0 B/0 for 82d35ab01095f590b40a2bc17e2059db in 483ms, sequenceid=88, compaction requested=true 2023-07-16 21:16:13,506 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.47 KB/3554, heapSize ~5.91 KB/6056, currentSize=0 B/0 for 1588230740 in 480ms, sequenceid=163, compaction requested=true 2023-07-16 21:16:13,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/recovered.edits/91.seqid, newMaxSeqId=91, maxSeqId=77 2023-07-16 21:16:13,524 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/recovered.edits/166.seqid, newMaxSeqId=166, maxSeqId=152 2023-07-16 21:16:13,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-16 21:16:13,526 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:13,526 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-16 21:16:13,526 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 82d35ab01095f590b40a2bc17e2059db: 2023-07-16 21:16:13,526 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3513): Adding 82d35ab01095f590b40a2bc17e2059db move to jenkins-hbase4.apache.org,44109,1689542165940 record at close sequenceid=88 2023-07-16 21:16:13,527 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-16 21:16:13,527 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-16 21:16:13,527 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3513): Adding 1588230740 move to jenkins-hbase4.apache.org,44109,1689542165940 record at close sequenceid=163 2023-07-16 21:16:13,528 DEBUG [PEWorker-2] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=133, ppid=130, state=RUNNABLE; CloseRegionProcedure 82d35ab01095f590b40a2bc17e2059db, server=jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:13,529 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 1588230740 2023-07-16 21:16:13,529 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:13,530 WARN [PEWorker-3] zookeeper.MetaTableLocator(225): Tried to set null ServerName in hbase:meta; skipping -- ServerName required 2023-07-16 21:16:13,532 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=135, resume processing ppid=132 2023-07-16 21:16:13,532 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=135, ppid=132, state=SUCCESS; CloseRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40937,1689542166090 in 659 msec 2023-07-16 21:16:13,533 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=132, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,44109,1689542165940; forceNewPlan=false, retain=false 2023-07-16 21:16:13,683 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,44109,1689542165940, state=OPENING 2023-07-16 21:16:13,685 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-16 21:16:13,685 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-16 21:16:13,685 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=132, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,44109,1689542165940}] 2023-07-16 21:16:13,841 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-16 21:16:13,841 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:16:13,843 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44109%2C1689542165940.meta, suffix=.meta, logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,44109,1689542165940, archiveDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs, maxLogs=32 2023-07-16 21:16:13,862 DEBUG [RS-EventLoopGroup-17-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK] 2023-07-16 21:16:13,862 DEBUG [RS-EventLoopGroup-17-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK] 2023-07-16 21:16:13,862 DEBUG [RS-EventLoopGroup-17-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK] 2023-07-16 21:16:13,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] procedure.ProcedureSyncWait(216): waitFor pid=130 2023-07-16 21:16:13,872 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,44109,1689542165940/jenkins-hbase4.apache.org%2C44109%2C1689542165940.meta.1689542173843.meta 2023-07-16 21:16:13,872 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46257,DS-d4b2355c-0e54-4fd0-938c-57c9d3b772d1,DISK], DatanodeInfoWithStorage[127.0.0.1:36325,DS-d9831650-bcbe-4b26-9a48-5cc502a954ad,DISK], DatanodeInfoWithStorage[127.0.0.1:43471,DS-2842b6f5-eab0-4fcc-8147-9d01f653cd00,DISK]] 2023-07-16 21:16:13,872 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:16:13,873 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-16 21:16:13,873 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-16 21:16:13,873 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-16 21:16:13,873 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-16 21:16:13,873 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:16:13,873 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-16 21:16:13,873 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-16 21:16:13,875 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-16 21:16:13,876 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info 2023-07-16 21:16:13,877 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info 2023-07-16 21:16:13,877 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-16 21:16:13,893 INFO [StoreFileOpener-info-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c49957a900a44abbf01b64d8b340ddd 2023-07-16 21:16:13,893 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/8c49957a900a44abbf01b64d8b340ddd 2023-07-16 21:16:13,898 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/ca6165e191fc4d699c6c92331ab746d5 2023-07-16 21:16:13,903 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/fb397dc30b5940959a1c1af2e67ba90f 2023-07-16 21:16:13,903 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:16:13,903 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-16 21:16:13,904 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/rep_barrier 2023-07-16 21:16:13,904 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/rep_barrier 2023-07-16 21:16:13,905 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-16 21:16:13,912 INFO [StoreFileOpener-rep_barrier-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a387d081595450199b46921f8b69aeb 2023-07-16 21:16:13,912 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/rep_barrier/7a387d081595450199b46921f8b69aeb 2023-07-16 21:16:13,912 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:16:13,913 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-16 21:16:13,913 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table 2023-07-16 21:16:13,914 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table 2023-07-16 21:16:13,914 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-16 21:16:13,921 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table/39ca1cf509684b6ab736d32a3c68a7be 2023-07-16 21:16:13,927 INFO [StoreFileOpener-table-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e6b7923a5b4640e084b40f28aa6202c9 2023-07-16 21:16:13,928 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/table/e6b7923a5b4640e084b40f28aa6202c9 2023-07-16 21:16:13,928 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:16:13,929 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740 2023-07-16 21:16:13,930 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740 2023-07-16 21:16:13,932 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-16 21:16:13,934 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-16 21:16:13,935 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=167; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=9911022880, jitterRate=-0.07696406543254852}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-16 21:16:13,935 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-16 21:16:13,935 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=136, masterSystemTime=1689542173837 2023-07-16 21:16:13,937 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-16 21:16:13,940 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-07-16 21:16:13,940 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-16 21:16:13,940 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-16 21:16:13,941 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,44109,1689542165940, state=OPEN 2023-07-16 21:16:13,942 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-16 21:16:13,942 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-16 21:16:13,944 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=130 updating hbase:meta row=82d35ab01095f590b40a2bc17e2059db, regionState=CLOSED 2023-07-16 21:16:13,944 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=73c0efa65c224047ddc5ff1870c16b5e, regionState=CLOSED 2023-07-16 21:16:13,944 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1689542173944"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542173944"}]},"ts":"1689542173944"} 2023-07-16 21:16:13,944 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689542173944"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689542173944"}]},"ts":"1689542173944"} 2023-07-16 21:16:13,944 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 27055 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-07-16 21:16:13,945 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40937] ipc.CallRunner(144): callId: 61 service: ClientService methodName: Mutate size: 213 connection: 172.31.14.131:55208 deadline: 1689542233944, exception=org.apache.hadoop.hbase.exceptions.RegionMovedException: Region moved to: hostname=jenkins-hbase4.apache.org port=44109 startCode=1689542165940. As of locationSeqNum=163. 2023-07-16 21:16:13,945 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40937] ipc.CallRunner(144): callId: 62 service: ClientService methodName: Mutate size: 217 connection: 172.31.14.131:55208 deadline: 1689542233945, exception=org.apache.hadoop.hbase.exceptions.RegionMovedException: Region moved to: hostname=jenkins-hbase4.apache.org port=44109 startCode=1689542165940. As of locationSeqNum=163. 2023-07-16 21:16:13,951 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=132 2023-07-16 21:16:13,951 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=132, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,44109,1689542165940 in 257 msec 2023-07-16 21:16:13,952 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.HStore(1912): 1588230740/info is initiating minor compaction (all files) 2023-07-16 21:16:13,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, REOPEN/MOVE in 1.0860 sec 2023-07-16 21:16:13,952 INFO [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1588230740/info in hbase:meta,,1.1588230740 2023-07-16 21:16:13,952 INFO [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/8c49957a900a44abbf01b64d8b340ddd, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/ca6165e191fc4d699c6c92331ab746d5, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/fb397dc30b5940959a1c1af2e67ba90f] into tmpdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp, totalSize=26.4 K 2023-07-16 21:16:13,953 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] compactions.Compactor(207): Compacting 8c49957a900a44abbf01b64d8b340ddd, keycount=52, bloomtype=NONE, size=10.5 K, encoding=NONE, compression=NONE, seqNum=134, earliestPutTs=1689542137097 2023-07-16 21:16:13,954 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] compactions.Compactor(207): Compacting ca6165e191fc4d699c6c92331ab746d5, keycount=26, bloomtype=NONE, size=7.7 K, encoding=NONE, compression=NONE, seqNum=149, earliestPutTs=1689542158542 2023-07-16 21:16:13,954 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] compactions.Compactor(207): Compacting fb397dc30b5940959a1c1af2e67ba90f, keycount=30, bloomtype=NONE, size=8.2 K, encoding=NONE, compression=NONE, seqNum=163, earliestPutTs=1689542171221 2023-07-16 21:16:13,972 INFO [RS:0;jenkins-hbase4:44109-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1588230740#info#compaction#12 average throughput is 5.62 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-16 21:16:13,993 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp/info/375fb68f341e41ebb7a7df5206ba0212 as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/375fb68f341e41ebb7a7df5206ba0212 2023-07-16 21:16:14,009 INFO [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1588230740/info of 1588230740 into 375fb68f341e41ebb7a7df5206ba0212(size=10.6 K), total size for store is 10.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-16 21:16:14,009 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1588230740: 2023-07-16 21:16:14,010 INFO [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=hbase:meta,,1.1588230740, storeName=1588230740/info, priority=13, startTime=1689542173937; duration=0sec 2023-07-16 21:16:14,010 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-16 21:16:14,051 DEBUG [PEWorker-4] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:16:14,052 INFO [RS-EventLoopGroup-14-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48888, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:16:14,055 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=130 2023-07-16 21:16:14,055 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=130, state=SUCCESS; CloseRegionProcedure 82d35ab01095f590b40a2bc17e2059db, server=jenkins-hbase4.apache.org,36613,1689542166251 in 1.1890 sec 2023-07-16 21:16:14,056 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=130, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,44109,1689542165940; forceNewPlan=false, retain=false 2023-07-16 21:16:14,056 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-07-16 21:16:14,056 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; CloseRegionProcedure 73c0efa65c224047ddc5ff1870c16b5e, server=jenkins-hbase4.apache.org,40937,1689542166090 in 1.1860 sec 2023-07-16 21:16:14,057 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=131, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,44109,1689542165940; forceNewPlan=false, retain=false 2023-07-16 21:16:14,057 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=130 updating hbase:meta row=82d35ab01095f590b40a2bc17e2059db, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:14,057 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1689542174057"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542174057"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542174057"}]},"ts":"1689542174057"} 2023-07-16 21:16:14,058 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=73c0efa65c224047ddc5ff1870c16b5e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:14,058 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689542174058"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1689542174058"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689542174058"}]},"ts":"1689542174058"} 2023-07-16 21:16:14,059 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=137, ppid=130, state=RUNNABLE; OpenRegionProcedure 82d35ab01095f590b40a2bc17e2059db, server=jenkins-hbase4.apache.org,44109,1689542165940}] 2023-07-16 21:16:14,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=131, state=RUNNABLE; OpenRegionProcedure 73c0efa65c224047ddc5ff1870c16b5e, server=jenkins-hbase4.apache.org,44109,1689542165940}] 2023-07-16 21:16:14,214 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:14,215 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 73c0efa65c224047ddc5ff1870c16b5e, NAME => 'hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:16:14,215 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:14,215 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:16:14,215 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:14,215 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:14,217 INFO [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:14,218 DEBUG [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/info 2023-07-16 21:16:14,219 DEBUG [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/info 2023-07-16 21:16:14,219 INFO [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 73c0efa65c224047ddc5ff1870c16b5e columnFamilyName info 2023-07-16 21:16:14,225 INFO [StoreFileOpener-info-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7aca6aa2367450cb461f1ac0a602f75 2023-07-16 21:16:14,225 DEBUG [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/info/e7aca6aa2367450cb461f1ac0a602f75 2023-07-16 21:16:14,225 INFO [StoreOpener-73c0efa65c224047ddc5ff1870c16b5e-1] regionserver.HStore(310): Store=73c0efa65c224047ddc5ff1870c16b5e/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:16:14,226 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:14,227 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:14,230 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:14,230 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 73c0efa65c224047ddc5ff1870c16b5e; next sequenceid=21; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=9431474080, jitterRate=-0.12162552773952484}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:16:14,230 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 73c0efa65c224047ddc5ff1870c16b5e: 2023-07-16 21:16:14,231 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e., pid=138, masterSystemTime=1689542174211 2023-07-16 21:16:14,233 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:14,233 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:14,233 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:14,233 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=73c0efa65c224047ddc5ff1870c16b5e, regionState=OPEN, openSeqNum=21, regionLocation=jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:14,234 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 82d35ab01095f590b40a2bc17e2059db, NAME => 'hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.', STARTKEY => '', ENDKEY => ''} 2023-07-16 21:16:14,234 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689542174233"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542174233"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542174233"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542174233"}]},"ts":"1689542174233"} 2023-07-16 21:16:14,234 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-16 21:16:14,234 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. service=MultiRowMutationService 2023-07-16 21:16:14,234 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:rsgroup successfully. 2023-07-16 21:16:14,234 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table rsgroup 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:14,234 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-16 21:16:14,234 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:14,234 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:14,236 INFO [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family m of region 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:14,238 DEBUG [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m 2023-07-16 21:16:14,238 DEBUG [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m 2023-07-16 21:16:14,238 INFO [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 82d35ab01095f590b40a2bc17e2059db columnFamilyName m 2023-07-16 21:16:14,238 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=131 2023-07-16 21:16:14,238 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=131, state=SUCCESS; OpenRegionProcedure 73c0efa65c224047ddc5ff1870c16b5e, server=jenkins-hbase4.apache.org,44109,1689542165940 in 177 msec 2023-07-16 21:16:14,242 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=131, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=73c0efa65c224047ddc5ff1870c16b5e, REOPEN/MOVE in 1.3740 sec 2023-07-16 21:16:14,245 DEBUG [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/23d165da51744f1692de8df99d4f87ef 2023-07-16 21:16:14,253 INFO [StoreFileOpener-m-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 95305049645a40afbe27bf61c4b28d28 2023-07-16 21:16:14,253 DEBUG [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/95305049645a40afbe27bf61c4b28d28 2023-07-16 21:16:14,258 INFO [StoreFileOpener-m-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d185255a3a8e4435a58d7c4f27f39d36 2023-07-16 21:16:14,258 DEBUG [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] regionserver.HStore(539): loaded hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/d185255a3a8e4435a58d7c4f27f39d36 2023-07-16 21:16:14,258 INFO [StoreOpener-82d35ab01095f590b40a2bc17e2059db-1] regionserver.HStore(310): Store=82d35ab01095f590b40a2bc17e2059db/m, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-16 21:16:14,259 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:14,260 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:14,263 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:14,264 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 82d35ab01095f590b40a2bc17e2059db; next sequenceid=92; org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy@6728939e, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-16 21:16:14,264 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 82d35ab01095f590b40a2bc17e2059db: 2023-07-16 21:16:14,265 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db., pid=137, masterSystemTime=1689542174211 2023-07-16 21:16:14,265 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-16 21:16:14,266 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-07-16 21:16:14,267 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 16352 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-07-16 21:16:14,268 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.HStore(1912): 82d35ab01095f590b40a2bc17e2059db/m is initiating minor compaction (all files) 2023-07-16 21:16:14,268 INFO [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 82d35ab01095f590b40a2bc17e2059db/m in hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:14,268 INFO [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/d185255a3a8e4435a58d7c4f27f39d36, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/23d165da51744f1692de8df99d4f87ef, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/95305049645a40afbe27bf61c4b28d28] into tmpdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/.tmp, totalSize=16.0 K 2023-07-16 21:16:14,268 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:14,268 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:14,268 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=130 updating hbase:meta row=82d35ab01095f590b40a2bc17e2059db, regionState=OPEN, openSeqNum=92, regionLocation=jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:14,268 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] compactions.Compactor(207): Compacting d185255a3a8e4435a58d7c4f27f39d36, keycount=21, bloomtype=ROW, size=5.7 K, encoding=NONE, compression=NONE, seqNum=67, earliestPutTs=1689542149705 2023-07-16 21:16:14,268 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1689542174268"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1689542174268"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689542174268"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689542174268"}]},"ts":"1689542174268"} 2023-07-16 21:16:14,269 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] compactions.Compactor(207): Compacting 23d165da51744f1692de8df99d4f87ef, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=74, earliestPutTs=1689542162758 2023-07-16 21:16:14,269 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] compactions.Compactor(207): Compacting 95305049645a40afbe27bf61c4b28d28, keycount=5, bloomtype=ROW, size=5.3 K, encoding=NONE, compression=NONE, seqNum=88, earliestPutTs=1689542172856 2023-07-16 21:16:14,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=130 2023-07-16 21:16:14,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=130, state=SUCCESS; OpenRegionProcedure 82d35ab01095f590b40a2bc17e2059db, server=jenkins-hbase4.apache.org,44109,1689542165940 in 211 msec 2023-07-16 21:16:14,273 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=130, state=SUCCESS; TransitRegionStateProcedure table=hbase:rsgroup, region=82d35ab01095f590b40a2bc17e2059db, REOPEN/MOVE in 1.4110 sec 2023-07-16 21:16:14,291 INFO [RS:0;jenkins-hbase4:44109-shortCompactions-0] throttle.PressureAwareThroughputController(145): 82d35ab01095f590b40a2bc17e2059db#m#compaction#13 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-16 21:16:14,329 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/.tmp/m/abe0a71c7ef94a199caabc930300f0f9 as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/abe0a71c7ef94a199caabc930300f0f9 2023-07-16 21:16:14,335 INFO [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 82d35ab01095f590b40a2bc17e2059db/m of 82d35ab01095f590b40a2bc17e2059db into abe0a71c7ef94a199caabc930300f0f9(size=5.3 K), total size for store is 5.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-16 21:16:14,335 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 82d35ab01095f590b40a2bc17e2059db: 2023-07-16 21:16:14,335 INFO [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db., storeName=82d35ab01095f590b40a2bc17e2059db/m, priority=13, startTime=1689542174265; duration=0sec 2023-07-16 21:16:14,335 DEBUG [RS:0;jenkins-hbase4:44109-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-16 21:16:14,872 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminServer(261): All regions from [jenkins-hbase4.apache.org,34749,1689542172394, jenkins-hbase4.apache.org,36613,1689542166251, jenkins-hbase4.apache.org,40937,1689542166090] are moved back to default 2023-07-16 21:16:14,872 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminServer(438): Move servers done: default => Group_testClearDeadServers_1449045704 2023-07-16 21:16:14,872 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:16:14,874 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36613] ipc.CallRunner(144): callId: 4 service: ClientService methodName: Scan size: 136 connection: 172.31.14.131:43804 deadline: 1689542234874, exception=org.apache.hadoop.hbase.exceptions.RegionMovedException: Region moved to: hostname=jenkins-hbase4.apache.org port=44109 startCode=1689542165940. As of locationSeqNum=88. 2023-07-16 21:16:14,978 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40937] ipc.CallRunner(144): callId: 5 service: ClientService methodName: Get size: 88 connection: 172.31.14.131:36536 deadline: 1689542234978, exception=org.apache.hadoop.hbase.exceptions.RegionMovedException: Region moved to: hostname=jenkins-hbase4.apache.org port=44109 startCode=1689542165940. As of locationSeqNum=163. 2023-07-16 21:16:15,081 DEBUG [hconnection-0x2672383d-shared-pool-3] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-16 21:16:15,082 INFO [RS-EventLoopGroup-14-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48896, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-16 21:16:15,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:16:15,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:16:15,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=Group_testClearDeadServers_1449045704 2023-07-16 21:16:15,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:16:15,096 DEBUG [Listener at localhost/43605] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-16 21:16:15,098 INFO [RS-EventLoopGroup-17-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51466, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-16 21:16:15,098 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34749] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34749,1689542172394' ***** 2023-07-16 21:16:15,098 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34749] regionserver.HRegionServer(2311): STOPPED: Called by admin client hconnection-0x34944faa 2023-07-16 21:16:15,098 INFO [RS:3;jenkins-hbase4:34749] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:16:15,101 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:16:15,102 INFO [RS:3;jenkins-hbase4:34749] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@7cfd6419{regionserver,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:16:15,102 INFO [RS:3;jenkins-hbase4:34749] server.AbstractConnector(383): Stopped ServerConnector@2028dbf3{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:16:15,102 INFO [RS:3;jenkins-hbase4:34749] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:16:15,103 INFO [RS:3;jenkins-hbase4:34749] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@2cc42138{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:16:15,104 INFO [RS:3;jenkins-hbase4:34749] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@6e5a7c27{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:16:15,105 INFO [RS:3;jenkins-hbase4:34749] regionserver.HeapMemoryManager(220): Stopping 2023-07-16 21:16:15,105 INFO [RS:3;jenkins-hbase4:34749] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-16 21:16:15,105 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-16 21:16:15,105 INFO [RS:3;jenkins-hbase4:34749] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-16 21:16:15,105 INFO [RS:3;jenkins-hbase4:34749] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34749,1689542172394 2023-07-16 21:16:15,105 DEBUG [RS:3;jenkins-hbase4:34749] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x153625b0 to 127.0.0.1:63917 2023-07-16 21:16:15,105 DEBUG [RS:3;jenkins-hbase4:34749] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:15,105 INFO [RS:3;jenkins-hbase4:34749] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34749,1689542172394; all regions closed. 2023-07-16 21:16:15,121 DEBUG [RS:3;jenkins-hbase4:34749] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs 2023-07-16 21:16:15,121 INFO [RS:3;jenkins-hbase4:34749] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34749%2C1689542172394:(num 1689542172787) 2023-07-16 21:16:15,121 DEBUG [RS:3;jenkins-hbase4:34749] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:15,121 INFO [RS:3;jenkins-hbase4:34749] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:15,121 INFO [RS:3;jenkins-hbase4:34749] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-16 21:16:15,121 INFO [RS:3;jenkins-hbase4:34749] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-16 21:16:15,121 INFO [RS:3;jenkins-hbase4:34749] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-16 21:16:15,121 INFO [RS:3;jenkins-hbase4:34749] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-16 21:16:15,122 INFO [RS:3;jenkins-hbase4:34749] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34749 2023-07-16 21:16:15,122 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:16:15,152 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:15,211 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34749,1689542172394 2023-07-16 21:16:15,211 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:15,211 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34749,1689542172394 2023-07-16 21:16:15,211 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:15,211 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34749-0x1016ffd80780028, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34749,1689542172394 2023-07-16 21:16:15,211 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34749-0x1016ffd80780028, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:15,211 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:15,211 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34749,1689542172394 2023-07-16 21:16:15,211 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:15,212 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34749,1689542172394] 2023-07-16 21:16:15,212 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34749,1689542172394; numProcessing=1 2023-07-16 21:16:15,213 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34749,1689542172394 already deleted, retry=false 2023-07-16 21:16:15,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:15,213 INFO [RegionServerTracker-0] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,34749,1689542172394 on jenkins-hbase4.apache.org,33973,1689542165780 2023-07-16 21:16:15,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:15,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:15,214 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:15,214 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:15,214 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:15,214 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,34749,1689542172394 znode expired, triggering replicatorRemoved event 2023-07-16 21:16:15,214 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:15,214 DEBUG [RegionServerTracker-0] procedure2.ProcedureExecutor(1029): Stored pid=139, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,34749,1689542172394, splitWal=true, meta=false 2023-07-16 21:16:15,214 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:15,215 INFO [RegionServerTracker-0] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=139 for jenkins-hbase4.apache.org,34749,1689542172394 (carryingMeta=false) jenkins-hbase4.apache.org,34749,1689542172394/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@3e458899[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-16 21:16:15,215 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,34749,1689542172394 znode expired, triggering replicatorRemoved event 2023-07-16 21:16:15,215 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:16:15,224 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36613] ipc.CallRunner(144): callId: 70 service: ClientService methodName: ExecService size: 574 connection: 172.31.14.131:43792 deadline: 1689542235224, exception=org.apache.hadoop.hbase.exceptions.RegionMovedException: Region moved to: hostname=jenkins-hbase4.apache.org port=44109 startCode=1689542165940. As of locationSeqNum=88. 2023-07-16 21:16:15,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:15,226 INFO [PEWorker-5] procedure.ServerCrashProcedure(161): Start pid=139, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34749,1689542172394, splitWal=true, meta=false 2023-07-16 21:16:15,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:15,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:15,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:15,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:15,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:15,228 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,34749,1689542172394 znode expired, triggering replicatorRemoved event 2023-07-16 21:16:15,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:15,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:15,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:15,230 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:15,230 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,34749,1689542172394 had 0 regions 2023-07-16 21:16:15,232 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=139, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34749,1689542172394, splitWal=true, meta=false, isMeta: false 2023-07-16 21:16:15,233 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34749,1689542172394-splitting 2023-07-16 21:16:15,234 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34749,1689542172394-splitting dir is empty, no logs to split. 2023-07-16 21:16:15,234 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,34749,1689542172394 WAL count=0, meta=false 2023-07-16 21:16:15,236 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34749,1689542172394-splitting dir is empty, no logs to split. 2023-07-16 21:16:15,236 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,34749,1689542172394 WAL count=0, meta=false 2023-07-16 21:16:15,236 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34749,1689542172394 WAL splitting is done? wals=0, meta=false 2023-07-16 21:16:15,241 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,34749,1689542172394 failed, ignore...File hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,34749,1689542172394-splitting does not exist. 2023-07-16 21:16:15,245 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,34749,1689542172394 after splitting done 2023-07-16 21:16:15,245 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,34749,1689542172394 from processing; numProcessing=0 2023-07-16 21:16:15,246 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=139, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34749,1689542172394, splitWal=true, meta=false in 32 msec 2023-07-16 21:16:15,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(2362): Client=jenkins//172.31.14.131 clear dead region servers. 2023-07-16 21:16:15,312 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34749-0x1016ffd80780028, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:15,312 INFO [RS:3;jenkins-hbase4:34749] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34749,1689542172394; zookeeper connection closed. 2023-07-16 21:16:15,312 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:34749-0x1016ffd80780028, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:15,313 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3474ddbb] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3474ddbb 2023-07-16 21:16:15,333 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:15,333 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/Group_testClearDeadServers_1449045704 2023-07-16 21:16:15,333 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:16:15,334 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:16:15,336 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 1 2023-07-16 21:16:15,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:15,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/Group_testClearDeadServers_1449045704 2023-07-16 21:16:15,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:16:15,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:16:15,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminServer(609): Remove decommissioned servers [jenkins-hbase4.apache.org:34749] from RSGroup done 2023-07-16 21:16:15,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=Group_testClearDeadServers_1449045704 2023-07-16 21:16:15,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:16:15,343 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40937] ipc.CallRunner(144): callId: 74 service: ClientService methodName: Scan size: 146 connection: 172.31.14.131:55208 deadline: 1689542235343, exception=org.apache.hadoop.hbase.exceptions.RegionMovedException: Region moved to: hostname=jenkins-hbase4.apache.org port=44109 startCode=1689542165940. As of locationSeqNum=18. 2023-07-16 21:16:15,452 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:16:15,452 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:16:15,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:16:15,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:16:15,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:16:15,454 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:36613, jenkins-hbase4.apache.org:40937] to rsgroup default 2023-07-16 21:16:15,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:15,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/Group_testClearDeadServers_1449045704 2023-07-16 21:16:15,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:16:15,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-16 21:16:15,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminServer(286): Moving 0 region(s) to group Group_testClearDeadServers_1449045704, current retry=0 2023-07-16 21:16:15,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminServer(261): All regions from [jenkins-hbase4.apache.org,36613,1689542166251, jenkins-hbase4.apache.org,40937,1689542166090] are moved back to Group_testClearDeadServers_1449045704 2023-07-16 21:16:15,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminServer(438): Move servers done: Group_testClearDeadServers_1449045704 => default 2023-07-16 21:16:15,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:16:15,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup Group_testClearDeadServers_1449045704 2023-07-16 21:16:15,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:15,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:16:15,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 5 2023-07-16 21:16:15,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:16:15,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-16 21:16:15,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-16 21:16:15,474 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-16 21:16:15,474 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-16 21:16:15,474 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-16 21:16:15,475 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-16 21:16:15,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:15,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-16 21:16:15,479 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-16 21:16:15,482 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase(152): Restoring servers: 1 2023-07-16 21:16:15,494 INFO [Listener at localhost/43605] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-16 21:16:15,494 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:15,494 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:15,494 INFO [Listener at localhost/43605] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-16 21:16:15,494 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-16 21:16:15,494 INFO [Listener at localhost/43605] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-16 21:16:15,494 INFO [Listener at localhost/43605] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-16 21:16:15,495 INFO [Listener at localhost/43605] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41705 2023-07-16 21:16:15,496 INFO [Listener at localhost/43605] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-16 21:16:15,498 DEBUG [Listener at localhost/43605] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-16 21:16:15,499 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:16:15,500 INFO [Listener at localhost/43605] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-16 21:16:15,501 INFO [Listener at localhost/43605] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41705 connecting to ZooKeeper ensemble=127.0.0.1:63917 2023-07-16 21:16:15,503 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:417050x0, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-16 21:16:15,506 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41705-0x1016ffd8078002a connected 2023-07-16 21:16:15,506 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(162): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-16 21:16:15,506 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(162): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/running 2023-07-16 21:16:15,507 DEBUG [Listener at localhost/43605] zookeeper.ZKUtil(164): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-16 21:16:15,510 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41705 2023-07-16 21:16:15,510 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41705 2023-07-16 21:16:15,511 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41705 2023-07-16 21:16:15,515 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41705 2023-07-16 21:16:15,515 DEBUG [Listener at localhost/43605] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41705 2023-07-16 21:16:15,517 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-16 21:16:15,517 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-16 21:16:15,517 INFO [Listener at localhost/43605] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-16 21:16:15,518 INFO [Listener at localhost/43605] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-16 21:16:15,518 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-16 21:16:15,518 INFO [Listener at localhost/43605] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-16 21:16:15,519 INFO [Listener at localhost/43605] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-16 21:16:15,519 INFO [Listener at localhost/43605] http.HttpServer(1146): Jetty bound to port 45817 2023-07-16 21:16:15,519 INFO [Listener at localhost/43605] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-16 21:16:15,521 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:15,522 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@a448a54{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,AVAILABLE} 2023-07-16 21:16:15,522 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:15,522 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@477e1f15{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,AVAILABLE} 2023-07-16 21:16:15,640 INFO [Listener at localhost/43605] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-16 21:16:15,642 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-16 21:16:15,643 INFO [Listener at localhost/43605] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-16 21:16:15,643 INFO [Listener at localhost/43605] session.HouseKeeper(132): node0 Scavenging every 600000ms 2023-07-16 21:16:15,644 INFO [Listener at localhost/43605] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-16 21:16:15,645 INFO [Listener at localhost/43605] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@40562f60{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/java.io.tmpdir/jetty-0_0_0_0-45817-hbase-server-2_4_18-SNAPSHOT_jar-_-any-6929213885368811796/webapp/,AVAILABLE}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:16:15,647 INFO [Listener at localhost/43605] server.AbstractConnector(333): Started ServerConnector@5fa6b3cb{HTTP/1.1, (http/1.1)}{0.0.0.0:45817} 2023-07-16 21:16:15,648 INFO [Listener at localhost/43605] server.Server(415): Started @49512ms 2023-07-16 21:16:15,651 INFO [RS:4;jenkins-hbase4:41705] regionserver.HRegionServer(951): ClusterId : bc8d5a8a-d305-489f-bc1d-b95b98296a6c 2023-07-16 21:16:15,657 DEBUG [RS:4;jenkins-hbase4:41705] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-16 21:16:15,662 DEBUG [RS:4;jenkins-hbase4:41705] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-16 21:16:15,662 DEBUG [RS:4;jenkins-hbase4:41705] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-16 21:16:15,664 DEBUG [RS:4;jenkins-hbase4:41705] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-16 21:16:15,665 DEBUG [RS:4;jenkins-hbase4:41705] zookeeper.ReadOnlyZKClient(139): Connect 0x2bebdbb8 to 127.0.0.1:63917 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-16 21:16:15,670 DEBUG [RS:4;jenkins-hbase4:41705] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@252c20b5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-16 21:16:15,670 DEBUG [RS:4;jenkins-hbase4:41705] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2113b0c8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:16:15,680 DEBUG [RS:4;jenkins-hbase4:41705] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:4;jenkins-hbase4:41705 2023-07-16 21:16:15,680 INFO [RS:4;jenkins-hbase4:41705] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-16 21:16:15,680 INFO [RS:4;jenkins-hbase4:41705] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-16 21:16:15,680 DEBUG [RS:4;jenkins-hbase4:41705] regionserver.HRegionServer(1022): About to register with Master. 2023-07-16 21:16:15,681 INFO [RS:4;jenkins-hbase4:41705] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33973,1689542165780 with isa=jenkins-hbase4.apache.org/172.31.14.131:41705, startcode=1689542175493 2023-07-16 21:16:15,681 DEBUG [RS:4;jenkins-hbase4:41705] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-16 21:16:15,684 INFO [RS-EventLoopGroup-13-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50257, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.12 (auth:SIMPLE), service=RegionServerStatusService 2023-07-16 21:16:15,684 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33973] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41705,1689542175493 2023-07-16 21:16:15,684 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-16 21:16:15,685 DEBUG [RS:4;jenkins-hbase4:41705] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb 2023-07-16 21:16:15,685 DEBUG [RS:4;jenkins-hbase4:41705] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:35261 2023-07-16 21:16:15,685 DEBUG [RS:4;jenkins-hbase4:41705] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=38591 2023-07-16 21:16:15,686 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:15,686 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:15,686 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:15,686 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:15,686 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:15,687 DEBUG [RS:4;jenkins-hbase4:41705] zookeeper.ZKUtil(162): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41705,1689542175493 2023-07-16 21:16:15,687 WARN [RS:4;jenkins-hbase4:41705] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-16 21:16:15,687 INFO [RS:4;jenkins-hbase4:41705] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-16 21:16:15,687 DEBUG [RS:4;jenkins-hbase4:41705] regionserver.HRegionServer(1948): logDir=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/WALs/jenkins-hbase4.apache.org,41705,1689542175493 2023-07-16 21:16:15,687 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41705,1689542175493] 2023-07-16 21:16:15,688 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 2 2023-07-16 21:16:15,688 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41705,1689542175493 2023-07-16 21:16:15,688 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41705,1689542175493 2023-07-16 21:16:15,688 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41705,1689542175493 2023-07-16 21:16:15,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:15,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:15,689 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33973,1689542165780] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 4 2023-07-16 21:16:15,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:15,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:15,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:15,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:15,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:15,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:15,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:15,701 DEBUG [RS:4;jenkins-hbase4:41705] zookeeper.ZKUtil(162): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41705,1689542175493 2023-07-16 21:16:15,701 DEBUG [RS:4;jenkins-hbase4:41705] zookeeper.ZKUtil(162): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:15,701 DEBUG [RS:4;jenkins-hbase4:41705] zookeeper.ZKUtil(162): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:15,702 DEBUG [RS:4;jenkins-hbase4:41705] zookeeper.ZKUtil(162): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:15,702 DEBUG [RS:4;jenkins-hbase4:41705] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-16 21:16:15,703 INFO [RS:4;jenkins-hbase4:41705] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-16 21:16:15,704 INFO [RS:4;jenkins-hbase4:41705] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-16 21:16:15,708 INFO [RS:4;jenkins-hbase4:41705] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-16 21:16:15,708 INFO [RS:4;jenkins-hbase4:41705] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:15,708 INFO [RS:4;jenkins-hbase4:41705] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-16 21:16:15,710 INFO [RS:4;jenkins-hbase4:41705] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:15,711 DEBUG [RS:4;jenkins-hbase4:41705] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:15,711 DEBUG [RS:4;jenkins-hbase4:41705] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:15,711 DEBUG [RS:4;jenkins-hbase4:41705] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:15,711 DEBUG [RS:4;jenkins-hbase4:41705] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:15,711 DEBUG [RS:4;jenkins-hbase4:41705] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:15,711 DEBUG [RS:4;jenkins-hbase4:41705] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-16 21:16:15,711 DEBUG [RS:4;jenkins-hbase4:41705] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:15,711 DEBUG [RS:4;jenkins-hbase4:41705] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:15,711 DEBUG [RS:4;jenkins-hbase4:41705] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:15,711 DEBUG [RS:4;jenkins-hbase4:41705] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-16 21:16:15,715 INFO [RS:4;jenkins-hbase4:41705] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:15,715 INFO [RS:4;jenkins-hbase4:41705] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:15,716 INFO [RS:4;jenkins-hbase4:41705] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:15,733 INFO [RS:4;jenkins-hbase4:41705] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-16 21:16:15,734 INFO [RS:4;jenkins-hbase4:41705] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41705,1689542175493-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-16 21:16:15,762 INFO [RS:4;jenkins-hbase4:41705] regionserver.Replication(203): jenkins-hbase4.apache.org,41705,1689542175493 started 2023-07-16 21:16:15,763 INFO [RS:4;jenkins-hbase4:41705] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,41705,1689542175493, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41705, sessionid=0x1016ffd8078002a 2023-07-16 21:16:15,763 DEBUG [RS:4;jenkins-hbase4:41705] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-16 21:16:15,763 DEBUG [RS:4;jenkins-hbase4:41705] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41705,1689542175493 2023-07-16 21:16:15,763 DEBUG [RS:4;jenkins-hbase4:41705] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41705,1689542175493' 2023-07-16 21:16:15,763 DEBUG [RS:4;jenkins-hbase4:41705] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-16 21:16:15,763 DEBUG [RS:4;jenkins-hbase4:41705] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-16 21:16:15,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-16 21:16:15,764 DEBUG [RS:4;jenkins-hbase4:41705] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-16 21:16:15,764 DEBUG [RS:4;jenkins-hbase4:41705] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-16 21:16:15,764 DEBUG [RS:4;jenkins-hbase4:41705] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41705,1689542175493 2023-07-16 21:16:15,764 DEBUG [RS:4;jenkins-hbase4:41705] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41705,1689542175493' 2023-07-16 21:16:15,764 DEBUG [RS:4;jenkins-hbase4:41705] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-16 21:16:15,766 DEBUG [RS:4;jenkins-hbase4:41705] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-16 21:16:15,766 DEBUG [RS:4;jenkins-hbase4:41705] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-16 21:16:15,766 INFO [RS:4;jenkins-hbase4:41705] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-16 21:16:15,766 INFO [RS:4;jenkins-hbase4:41705] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-16 21:16:15,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-16 21:16:15,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-16 21:16:15,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-16 21:16:15,770 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-16 21:16:15,774 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:16:15,774 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:16:15,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:33973] to rsgroup master 2023-07-16 21:16:15,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:33973 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-16 21:16:15,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] ipc.CallRunner(144): callId: 104 service: MasterService methodName: ExecMasterService size: 118 connection: 172.31.14.131:45844 deadline: 1689543375776, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:33973 is either offline or it does not exist. 2023-07-16 21:16:15,777 WARN [Listener at localhost/43605] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:33973 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:33973 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-16 21:16:15,782 INFO [Listener at localhost/43605] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-16 21:16:15,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-16 21:16:15,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-16 21:16:15,783 INFO [Listener at localhost/43605] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:36613, jenkins-hbase4.apache.org:40937, jenkins-hbase4.apache.org:41705, jenkins-hbase4.apache.org:44109], Tables:[hbase:meta, hbase:quota, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-16 21:16:15,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-16 21:16:15,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33973] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-16 21:16:15,818 INFO [Listener at localhost/43605] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testClearDeadServers Thread=586 (was 564) - Thread LEAK? -, OpenFileDescriptor=924 (was 843) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=300 (was 300), ProcessCount=170 (was 170), AvailableMemoryMB=5082 (was 5234) 2023-07-16 21:16:15,818 WARN [Listener at localhost/43605] hbase.ResourceChecker(130): Thread=586 is superior to 500 2023-07-16 21:16:15,818 INFO [Listener at localhost/43605] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-16 21:16:15,819 INFO [Listener at localhost/43605] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-16 21:16:15,819 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x06aa2997 to 127.0.0.1:63917 2023-07-16 21:16:15,819 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:15,820 DEBUG [Listener at localhost/43605] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-16 21:16:15,820 DEBUG [Listener at localhost/43605] util.JVMClusterUtil(257): Found active master hash=231120047, stopped=false 2023-07-16 21:16:15,820 DEBUG [Listener at localhost/43605] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint 2023-07-16 21:16:15,820 DEBUG [Listener at localhost/43605] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver 2023-07-16 21:16:15,820 INFO [Listener at localhost/43605] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33973,1689542165780 2023-07-16 21:16:15,822 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-16 21:16:15,822 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-16 21:16:15,822 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-16 21:16:15,822 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-16 21:16:15,822 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:16:15,822 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-16 21:16:15,822 INFO [Listener at localhost/43605] procedure2.ProcedureExecutor(629): Stopping 2023-07-16 21:16:15,823 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:16:15,823 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:16:15,823 DEBUG [Listener at localhost/43605] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x184da964 to 127.0.0.1:63917 2023-07-16 21:16:15,823 DEBUG [Listener at localhost/43605] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:15,827 INFO [Listener at localhost/43605] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,44109,1689542165940' ***** 2023-07-16 21:16:15,827 INFO [Listener at localhost/43605] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-16 21:16:15,827 INFO [Listener at localhost/43605] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40937,1689542166090' ***** 2023-07-16 21:16:15,827 INFO [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:16:15,827 INFO [Listener at localhost/43605] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-16 21:16:15,827 INFO [Listener at localhost/43605] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36613,1689542166251' ***** 2023-07-16 21:16:15,827 INFO [Listener at localhost/43605] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-16 21:16:15,827 INFO [RS:1;jenkins-hbase4:40937] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:16:15,829 INFO [Listener at localhost/43605] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41705,1689542175493' ***** 2023-07-16 21:16:15,829 INFO [RS:2;jenkins-hbase4:36613] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:16:15,829 INFO [Listener at localhost/43605] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-16 21:16:15,829 INFO [RS:4;jenkins-hbase4:41705] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:16:15,836 INFO [RS:0;jenkins-hbase4:44109] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@65e3c6aa{regionserver,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:16:15,839 INFO [RS:0;jenkins-hbase4:44109] server.AbstractConnector(383): Stopped ServerConnector@505610f7{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:16:15,839 INFO [RS:0;jenkins-hbase4:44109] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:16:15,840 INFO [RS:2;jenkins-hbase4:36613] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@2c606038{regionserver,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:16:15,840 INFO [RS:1;jenkins-hbase4:40937] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@41ecdee7{regionserver,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:16:15,840 INFO [RS:4;jenkins-hbase4:41705] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@40562f60{regionserver,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/regionserver} 2023-07-16 21:16:15,841 INFO [RS:0;jenkins-hbase4:44109] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@6711f9c9{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:16:15,841 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-16 21:16:15,841 INFO [RS:1;jenkins-hbase4:40937] server.AbstractConnector(383): Stopped ServerConnector@4ede1e50{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:16:15,841 INFO [RS:1;jenkins-hbase4:40937] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:16:15,841 INFO [RS:4;jenkins-hbase4:41705] server.AbstractConnector(383): Stopped ServerConnector@5fa6b3cb{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:16:15,842 INFO [RS:4;jenkins-hbase4:41705] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:16:15,842 INFO [RS:0;jenkins-hbase4:44109] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@43905abf{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:16:15,843 INFO [RS:4;jenkins-hbase4:41705] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@477e1f15{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:16:15,843 INFO [RS:2;jenkins-hbase4:36613] server.AbstractConnector(383): Stopped ServerConnector@21d758b1{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:16:15,844 INFO [RS:4;jenkins-hbase4:41705] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@a448a54{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:16:15,842 INFO [RS:1;jenkins-hbase4:40937] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@f13ea19{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:16:15,844 INFO [RS:2;jenkins-hbase4:36613] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:16:15,845 INFO [RS:1;jenkins-hbase4:40937] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@6e1f5678{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:16:15,845 INFO [RS:2;jenkins-hbase4:36613] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@c3d8f8{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:16:15,846 INFO [RS:0;jenkins-hbase4:44109] regionserver.HeapMemoryManager(220): Stopping 2023-07-16 21:16:15,847 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-16 21:16:15,847 INFO [RS:0;jenkins-hbase4:44109] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-16 21:16:15,847 INFO [RS:0;jenkins-hbase4:44109] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-16 21:16:15,847 INFO [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(3305): Received CLOSE for 73c0efa65c224047ddc5ff1870c16b5e 2023-07-16 21:16:15,847 INFO [RS:2;jenkins-hbase4:36613] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@5f0f0e67{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:16:15,847 INFO [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(3305): Received CLOSE for 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:15,847 INFO [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(3305): Received CLOSE for 08c6c5dca9905cead995e8dedf4b2e19 2023-07-16 21:16:15,847 INFO [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:15,847 DEBUG [RS:0;jenkins-hbase4:44109] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3716accb to 127.0.0.1:63917 2023-07-16 21:16:15,847 DEBUG [RS:0;jenkins-hbase4:44109] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:15,847 INFO [RS:4;jenkins-hbase4:41705] regionserver.HeapMemoryManager(220): Stopping 2023-07-16 21:16:15,848 INFO [RS:4;jenkins-hbase4:41705] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-16 21:16:15,848 INFO [RS:4;jenkins-hbase4:41705] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-16 21:16:15,847 INFO [RS:0;jenkins-hbase4:44109] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-16 21:16:15,848 INFO [RS:0;jenkins-hbase4:44109] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-16 21:16:15,848 INFO [RS:0;jenkins-hbase4:44109] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-16 21:16:15,848 INFO [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-16 21:16:15,848 INFO [RS:4;jenkins-hbase4:41705] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41705,1689542175493 2023-07-16 21:16:15,848 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-16 21:16:15,848 DEBUG [RS:4;jenkins-hbase4:41705] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2bebdbb8 to 127.0.0.1:63917 2023-07-16 21:16:15,848 INFO [RS:2;jenkins-hbase4:36613] regionserver.HeapMemoryManager(220): Stopping 2023-07-16 21:16:15,848 DEBUG [RS:4;jenkins-hbase4:41705] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:15,848 INFO [RS:4;jenkins-hbase4:41705] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41705,1689542175493; all regions closed. 2023-07-16 21:16:15,848 DEBUG [RS:4;jenkins-hbase4:41705] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:15,848 INFO [RS:4;jenkins-hbase4:41705] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:15,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 73c0efa65c224047ddc5ff1870c16b5e, disabling compactions & flushes 2023-07-16 21:16:15,851 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:15,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:15,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. after waiting 0 ms 2023-07-16 21:16:15,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:15,852 INFO [RS:4;jenkins-hbase4:41705] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-16 21:16:15,852 INFO [RS:2;jenkins-hbase4:36613] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-16 21:16:15,852 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:16:15,852 INFO [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-16 21:16:15,852 INFO [RS:2;jenkins-hbase4:36613] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-16 21:16:15,852 INFO [RS:4;jenkins-hbase4:41705] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-16 21:16:15,852 INFO [RS:2;jenkins-hbase4:36613] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:15,852 DEBUG [RS:2;jenkins-hbase4:36613] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7ba085ac to 127.0.0.1:63917 2023-07-16 21:16:15,852 DEBUG [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(1478): Online Regions={73c0efa65c224047ddc5ff1870c16b5e=hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e., 1588230740=hbase:meta,,1.1588230740, 82d35ab01095f590b40a2bc17e2059db=hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db., 08c6c5dca9905cead995e8dedf4b2e19=hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19.} 2023-07-16 21:16:15,852 DEBUG [RS:2;jenkins-hbase4:36613] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:15,852 DEBUG [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(1504): Waiting on 08c6c5dca9905cead995e8dedf4b2e19, 1588230740, 73c0efa65c224047ddc5ff1870c16b5e, 82d35ab01095f590b40a2bc17e2059db 2023-07-16 21:16:15,852 INFO [RS:4;jenkins-hbase4:41705] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-16 21:16:15,852 INFO [RS:4;jenkins-hbase4:41705] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-16 21:16:15,853 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-16 21:16:15,852 INFO [RS:2;jenkins-hbase4:36613] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36613,1689542166251; all regions closed. 2023-07-16 21:16:15,853 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-16 21:16:15,853 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-16 21:16:15,852 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-16 21:16:15,853 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-16 21:16:15,853 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-16 21:16:15,853 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.31 KB heapSize=4.53 KB 2023-07-16 21:16:15,855 INFO [RS:1;jenkins-hbase4:40937] regionserver.HeapMemoryManager(220): Stopping 2023-07-16 21:16:15,856 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:15,860 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-16 21:16:15,857 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:15,857 INFO [RS:1;jenkins-hbase4:40937] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-16 21:16:15,857 INFO [RS:4;jenkins-hbase4:41705] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41705 2023-07-16 21:16:15,860 INFO [RS:1;jenkins-hbase4:40937] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-16 21:16:15,860 INFO [RS:1;jenkins-hbase4:40937] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:15,860 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-16 21:16:15,860 DEBUG [RS:1;jenkins-hbase4:40937] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x792f238d to 127.0.0.1:63917 2023-07-16 21:16:15,860 DEBUG [RS:1;jenkins-hbase4:40937] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:15,861 INFO [RS:1;jenkins-hbase4:40937] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40937,1689542166090; all regions closed. 2023-07-16 21:16:15,885 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:15,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/namespace/73c0efa65c224047ddc5ff1870c16b5e/recovered.edits/23.seqid, newMaxSeqId=23, maxSeqId=20 2023-07-16 21:16:15,888 DEBUG [RS:2;jenkins-hbase4:36613] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs 2023-07-16 21:16:15,888 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:15,888 INFO [RS:2;jenkins-hbase4:36613] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36613%2C1689542166251:(num 1689542167010) 2023-07-16 21:16:15,888 DEBUG [RS:2;jenkins-hbase4:36613] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:15,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 73c0efa65c224047ddc5ff1870c16b5e: 2023-07-16 21:16:15,888 INFO [RS:2;jenkins-hbase4:36613] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:15,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689542136992.73c0efa65c224047ddc5ff1870c16b5e. 2023-07-16 21:16:15,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 82d35ab01095f590b40a2bc17e2059db, disabling compactions & flushes 2023-07-16 21:16:15,889 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:15,889 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:15,889 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. after waiting 0 ms 2023-07-16 21:16:15,889 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:15,889 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 82d35ab01095f590b40a2bc17e2059db 1/1 column families, dataSize=2.05 KB heapSize=3.55 KB 2023-07-16 21:16:15,890 INFO [RS:2;jenkins-hbase4:36613] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-16 21:16:15,890 INFO [RS:2;jenkins-hbase4:36613] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-16 21:16:15,890 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:16:15,890 INFO [RS:2;jenkins-hbase4:36613] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-16 21:16:15,890 INFO [RS:2;jenkins-hbase4:36613] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-16 21:16:15,891 DEBUG [RS:1;jenkins-hbase4:40937] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs 2023-07-16 21:16:15,891 INFO [RS:1;jenkins-hbase4:40937] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40937%2C1689542166090.meta:.meta(num 1689542167033) 2023-07-16 21:16:15,894 INFO [RS:2;jenkins-hbase4:36613] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36613 2023-07-16 21:16:15,895 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-16 21:16:15,895 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-16 21:16:15,905 DEBUG [RS:1;jenkins-hbase4:40937] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs 2023-07-16 21:16:15,905 INFO [RS:1;jenkins-hbase4:40937] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40937%2C1689542166090:(num 1689542167000) 2023-07-16 21:16:15,905 DEBUG [RS:1;jenkins-hbase4:40937] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:15,906 INFO [RS:1;jenkins-hbase4:40937] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:15,907 INFO [RS:1;jenkins-hbase4:40937] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-16 21:16:15,907 INFO [RS:1;jenkins-hbase4:40937] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-16 21:16:15,907 INFO [RS:1;jenkins-hbase4:40937] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-16 21:16:15,907 INFO [RS:1;jenkins-hbase4:40937] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-16 21:16:15,907 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:16:15,908 INFO [RS:1;jenkins-hbase4:40937] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40937 2023-07-16 21:16:15,910 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.31 KB at sequenceid=176 (bloomFilter=false), to=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp/info/da76140856e7453ca71fb45c84c8936b 2023-07-16 21:16:15,917 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/.tmp/info/da76140856e7453ca71fb45c84c8936b as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/da76140856e7453ca71fb45c84c8936b 2023-07-16 21:16:15,920 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:15,924 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:15,924 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:15,924 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:15,925 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:15,925 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:15,925 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:15,925 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:15,925 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:15,925 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40937,1689542166090 2023-07-16 21:16:15,925 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:15,925 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:15,925 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:15,925 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36613,1689542166251 2023-07-16 21:16:15,925 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.05 KB at sequenceid=102 (bloomFilter=true), to=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/.tmp/m/aace5977e98a43be9cd2a11b5595e275 2023-07-16 21:16:15,926 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/da76140856e7453ca71fb45c84c8936b, entries=20, sequenceid=176, filesize=7.0 K 2023-07-16 21:16:15,928 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41705,1689542175493 2023-07-16 21:16:15,928 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41705,1689542175493 2023-07-16 21:16:15,929 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41705,1689542175493 2023-07-16 21:16:15,929 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:16:15,929 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41705,1689542175493 2023-07-16 21:16:15,929 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:16:15,929 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.31 KB/2366, heapSize ~4.02 KB/4112, currentSize=0 B/0 for 1588230740 in 76ms, sequenceid=176, compaction requested=false 2023-07-16 21:16:15,929 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-16 21:16:15,936 DEBUG [StoreCloser-hbase:meta,,1.1588230740-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/8c49957a900a44abbf01b64d8b340ddd, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/ca6165e191fc4d699c6c92331ab746d5, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/fb397dc30b5940959a1c1af2e67ba90f] to archive 2023-07-16 21:16:15,937 DEBUG [StoreCloser-hbase:meta,,1.1588230740-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-16 21:16:15,938 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aace5977e98a43be9cd2a11b5595e275 2023-07-16 21:16:15,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/.tmp/m/aace5977e98a43be9cd2a11b5595e275 as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/aace5977e98a43be9cd2a11b5595e275 2023-07-16 21:16:15,941 DEBUG [StoreCloser-hbase:meta,,1.1588230740-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/8c49957a900a44abbf01b64d8b340ddd to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/hbase/meta/1588230740/info/8c49957a900a44abbf01b64d8b340ddd 2023-07-16 21:16:15,944 DEBUG [StoreCloser-hbase:meta,,1.1588230740-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/ca6165e191fc4d699c6c92331ab746d5 to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/hbase/meta/1588230740/info/ca6165e191fc4d699c6c92331ab746d5 2023-07-16 21:16:15,946 DEBUG [StoreCloser-hbase:meta,,1.1588230740-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/info/fb397dc30b5940959a1c1af2e67ba90f to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/hbase/meta/1588230740/info/fb397dc30b5940959a1c1af2e67ba90f 2023-07-16 21:16:15,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aace5977e98a43be9cd2a11b5595e275 2023-07-16 21:16:15,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/aace5977e98a43be9cd2a11b5595e275, entries=4, sequenceid=102, filesize=5.3 K 2023-07-16 21:16:15,951 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.05 KB/2100, heapSize ~3.54 KB/3624, currentSize=0 B/0 for 82d35ab01095f590b40a2bc17e2059db in 62ms, sequenceid=102, compaction requested=false 2023-07-16 21:16:15,951 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-16 21:16:15,951 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-16 21:16:15,959 DEBUG [StoreCloser-hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/d185255a3a8e4435a58d7c4f27f39d36, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/23d165da51744f1692de8df99d4f87ef, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/95305049645a40afbe27bf61c4b28d28] to archive 2023-07-16 21:16:15,960 DEBUG [StoreCloser-hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-16 21:16:15,962 DEBUG [StoreCloser-hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/d185255a3a8e4435a58d7c4f27f39d36 to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/d185255a3a8e4435a58d7c4f27f39d36 2023-07-16 21:16:15,964 DEBUG [StoreCloser-hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/23d165da51744f1692de8df99d4f87ef to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/23d165da51744f1692de8df99d4f87ef 2023-07-16 21:16:15,966 DEBUG [StoreCloser-hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/95305049645a40afbe27bf61c4b28d28 to hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/archive/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/m/95305049645a40afbe27bf61c4b28d28 2023-07-16 21:16:15,994 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/rsgroup/82d35ab01095f590b40a2bc17e2059db/recovered.edits/105.seqid, newMaxSeqId=105, maxSeqId=91 2023-07-16 21:16:15,995 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-16 21:16:15,995 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/meta/1588230740/recovered.edits/179.seqid, newMaxSeqId=179, maxSeqId=166 2023-07-16 21:16:15,995 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:15,995 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 82d35ab01095f590b40a2bc17e2059db: 2023-07-16 21:16:15,995 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:rsgroup,,1689542137297.82d35ab01095f590b40a2bc17e2059db. 2023-07-16 21:16:15,996 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 08c6c5dca9905cead995e8dedf4b2e19, disabling compactions & flushes 2023-07-16 21:16:15,996 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:16:15,996 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:16:15,996 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. after waiting 0 ms 2023-07-16 21:16:15,996 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:16:15,996 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-16 21:16:15,996 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-16 21:16:15,996 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-16 21:16:15,996 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-16 21:16:15,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/data/hbase/quota/08c6c5dca9905cead995e8dedf4b2e19/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-07-16 21:16:16,000 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:16:16,000 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 08c6c5dca9905cead995e8dedf4b2e19: 2023-07-16 21:16:16,000 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:quota,,1689542159546.08c6c5dca9905cead995e8dedf4b2e19. 2023-07-16 21:16:16,028 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:16,029 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:40937-0x1016ffd8078001e, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:16,029 INFO [RS:1;jenkins-hbase4:40937] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40937,1689542166090; zookeeper connection closed. 2023-07-16 21:16:16,029 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3ac7cc2b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3ac7cc2b 2023-07-16 21:16:16,030 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,40937,1689542166090] 2023-07-16 21:16:16,030 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40937,1689542166090; numProcessing=1 2023-07-16 21:16:16,032 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,40937,1689542166090 already deleted, retry=false 2023-07-16 21:16:16,032 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,40937,1689542166090 expired; onlineServers=3 2023-07-16 21:16:16,032 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41705,1689542175493] 2023-07-16 21:16:16,032 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41705,1689542175493; numProcessing=2 2023-07-16 21:16:16,033 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41705,1689542175493 already deleted, retry=false 2023-07-16 21:16:16,033 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41705,1689542175493 expired; onlineServers=2 2023-07-16 21:16:16,033 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36613,1689542166251] 2023-07-16 21:16:16,033 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36613,1689542166251; numProcessing=3 2023-07-16 21:16:16,035 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36613,1689542166251 already deleted, retry=false 2023-07-16 21:16:16,035 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36613,1689542166251 expired; onlineServers=1 2023-07-16 21:16:16,052 INFO [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,44109,1689542165940; all regions closed. 2023-07-16 21:16:16,059 DEBUG [RS:0;jenkins-hbase4:44109] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs 2023-07-16 21:16:16,060 INFO [RS:0;jenkins-hbase4:44109] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C44109%2C1689542165940.meta:.meta(num 1689542173843) 2023-07-16 21:16:16,064 DEBUG [RS:0;jenkins-hbase4:44109] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/oldWALs 2023-07-16 21:16:16,064 INFO [RS:0;jenkins-hbase4:44109] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C44109%2C1689542165940:(num 1689542167020) 2023-07-16 21:16:16,064 DEBUG [RS:0;jenkins-hbase4:44109] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:16,064 INFO [RS:0;jenkins-hbase4:44109] regionserver.LeaseManager(133): Closed leases 2023-07-16 21:16:16,065 INFO [RS:0;jenkins-hbase4:44109] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-16 21:16:16,065 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:16:16,066 INFO [RS:0;jenkins-hbase4:44109] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44109 2023-07-16 21:16:16,069 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-16 21:16:16,069 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,44109,1689542165940 2023-07-16 21:16:16,070 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,44109,1689542165940] 2023-07-16 21:16:16,071 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,44109,1689542165940; numProcessing=4 2023-07-16 21:16:16,072 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,44109,1689542165940 already deleted, retry=false 2023-07-16 21:16:16,072 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,44109,1689542165940 expired; onlineServers=0 2023-07-16 21:16:16,072 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33973,1689542165780' ***** 2023-07-16 21:16:16,072 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-16 21:16:16,072 DEBUG [M:0;jenkins-hbase4:33973] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@9f919cf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-16 21:16:16,072 INFO [M:0;jenkins-hbase4:33973] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-16 21:16:16,076 INFO [M:0;jenkins-hbase4:33973] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@4d6821d7{master,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/master} 2023-07-16 21:16:16,076 INFO [M:0;jenkins-hbase4:33973] server.AbstractConnector(383): Stopped ServerConnector@14c4c302{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:16:16,076 INFO [M:0;jenkins-hbase4:33973] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-16 21:16:16,077 INFO [M:0;jenkins-hbase4:33973] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@78851778{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/hbase-server-2.4.18-SNAPSHOT.jar!/hbase-webapps/static,STOPPED} 2023-07-16 21:16:16,077 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-07-16 21:16:16,078 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-16 21:16:16,078 INFO [M:0;jenkins-hbase4:33973] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@2bc95537{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/hadoop.log.dir/,STOPPED} 2023-07-16 21:16:16,078 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-16 21:16:16,078 INFO [M:0;jenkins-hbase4:33973] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33973,1689542165780 2023-07-16 21:16:16,078 INFO [M:0;jenkins-hbase4:33973] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33973,1689542165780; all regions closed. 2023-07-16 21:16:16,078 DEBUG [M:0;jenkins-hbase4:33973] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-16 21:16:16,078 INFO [M:0;jenkins-hbase4:33973] master.HMaster(1491): Stopping master jetty server 2023-07-16 21:16:16,079 INFO [M:0;jenkins-hbase4:33973] server.AbstractConnector(383): Stopped ServerConnector@1933a51a{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-16 21:16:16,084 DEBUG [M:0;jenkins-hbase4:33973] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-16 21:16:16,084 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-16 21:16:16,084 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689542166671] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1689542166671,5,FailOnTimeoutGroup] 2023-07-16 21:16:16,084 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689542166670] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1689542166670,5,FailOnTimeoutGroup] 2023-07-16 21:16:16,084 DEBUG [M:0;jenkins-hbase4:33973] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-16 21:16:16,085 INFO [M:0;jenkins-hbase4:33973] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-16 21:16:16,085 INFO [M:0;jenkins-hbase4:33973] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-16 21:16:16,085 INFO [M:0;jenkins-hbase4:33973] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-16 21:16:16,085 DEBUG [M:0;jenkins-hbase4:33973] master.HMaster(1512): Stopping service threads 2023-07-16 21:16:16,085 INFO [M:0;jenkins-hbase4:33973] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-16 21:16:16,085 ERROR [M:0;jenkins-hbase4:33973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-16 21:16:16,085 INFO [M:0;jenkins-hbase4:33973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-16 21:16:16,085 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-16 21:16:16,086 DEBUG [M:0;jenkins-hbase4:33973] zookeeper.ZKUtil(398): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-07-16 21:16:16,086 WARN [M:0;jenkins-hbase4:33973] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-16 21:16:16,086 INFO [M:0;jenkins-hbase4:33973] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-16 21:16:16,086 INFO [M:0;jenkins-hbase4:33973] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-16 21:16:16,087 DEBUG [M:0;jenkins-hbase4:33973] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-16 21:16:16,087 INFO [M:0;jenkins-hbase4:33973] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:16:16,087 DEBUG [M:0;jenkins-hbase4:33973] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:16:16,087 DEBUG [M:0;jenkins-hbase4:33973] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-16 21:16:16,087 DEBUG [M:0;jenkins-hbase4:33973] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:16:16,087 INFO [M:0;jenkins-hbase4:33973] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=64.10 KB heapSize=78.93 KB 2023-07-16 21:16:16,113 INFO [M:0;jenkins-hbase4:33973] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=64.10 KB at sequenceid=1058 (bloomFilter=true), to=hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b9ec2c277b1a4ffa8b7777727aa77a25 2023-07-16 21:16:16,119 DEBUG [M:0;jenkins-hbase4:33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b9ec2c277b1a4ffa8b7777727aa77a25 as hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b9ec2c277b1a4ffa8b7777727aa77a25 2023-07-16 21:16:16,124 INFO [M:0;jenkins-hbase4:33973] regionserver.HStore(1080): Added hdfs://localhost:35261/user/jenkins/test-data/72e6aff4-50f6-60bc-7a72-107a4c5c03eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b9ec2c277b1a4ffa8b7777727aa77a25, entries=21, sequenceid=1058, filesize=7.8 K 2023-07-16 21:16:16,125 INFO [M:0;jenkins-hbase4:33973] regionserver.HRegion(2948): Finished flush of dataSize ~64.10 KB/65634, heapSize ~78.91 KB/80808, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 38ms, sequenceid=1058, compaction requested=true 2023-07-16 21:16:16,128 INFO [M:0;jenkins-hbase4:33973] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-16 21:16:16,128 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:16,128 DEBUG [M:0;jenkins-hbase4:33973] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-16 21:16:16,128 INFO [RS:4;jenkins-hbase4:41705] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41705,1689542175493; zookeeper connection closed. 2023-07-16 21:16:16,128 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:41705-0x1016ffd8078002a, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:16,129 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@65d081d2] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@65d081d2 2023-07-16 21:16:16,138 INFO [M:0;jenkins-hbase4:33973] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-16 21:16:16,138 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-16 21:16:16,139 INFO [M:0;jenkins-hbase4:33973] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33973 2023-07-16 21:16:16,142 DEBUG [M:0;jenkins-hbase4:33973] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,33973,1689542165780 already deleted, retry=false 2023-07-16 21:16:16,629 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:16,629 INFO [M:0;jenkins-hbase4:33973] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33973,1689542165780; zookeeper connection closed. 2023-07-16 21:16:16,629 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): master:33973-0x1016ffd8078001c, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:16,729 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:16,730 INFO [RS:0;jenkins-hbase4:44109] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,44109,1689542165940; zookeeper connection closed. 2023-07-16 21:16:16,730 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:44109-0x1016ffd8078001d, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:16,730 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1104d3a7] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1104d3a7 2023-07-16 21:16:16,830 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:16,830 INFO [RS:2;jenkins-hbase4:36613] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36613,1689542166251; zookeeper connection closed. 2023-07-16 21:16:16,830 DEBUG [Listener at localhost/43605-EventThread] zookeeper.ZKWatcher(600): regionserver:36613-0x1016ffd8078001f, quorum=127.0.0.1:63917, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-16 21:16:16,830 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1a76de] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1a76de 2023-07-16 21:16:16,830 INFO [Listener at localhost/43605] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 5 regionserver(s) complete 2023-07-16 21:16:16,831 WARN [Listener at localhost/43605] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-16 21:16:16,837 INFO [Listener at localhost/43605] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-16 21:16:16,945 WARN [BP-1938682392-172.31.14.131-1689542128252 heartbeating to localhost/127.0.0.1:35261] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-16 21:16:16,945 WARN [BP-1938682392-172.31.14.131-1689542128252 heartbeating to localhost/127.0.0.1:35261] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1938682392-172.31.14.131-1689542128252 (Datanode Uuid 9f822f71-4eeb-4eb3-b4c1-9f5b5890a878) service to localhost/127.0.0.1:35261 2023-07-16 21:16:16,947 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5/dfs/data/data5/current/BP-1938682392-172.31.14.131-1689542128252] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-16 21:16:16,947 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5/dfs/data/data6/current/BP-1938682392-172.31.14.131-1689542128252] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-16 21:16:16,949 WARN [Listener at localhost/43605] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-16 21:16:16,959 INFO [Listener at localhost/43605] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-16 21:16:17,064 WARN [BP-1938682392-172.31.14.131-1689542128252 heartbeating to localhost/127.0.0.1:35261] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-16 21:16:17,064 WARN [BP-1938682392-172.31.14.131-1689542128252 heartbeating to localhost/127.0.0.1:35261] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1938682392-172.31.14.131-1689542128252 (Datanode Uuid 86c0afe4-f4d7-43f2-9612-8fecb0477082) service to localhost/127.0.0.1:35261 2023-07-16 21:16:17,065 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5/dfs/data/data3/current/BP-1938682392-172.31.14.131-1689542128252] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-16 21:16:17,065 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5/dfs/data/data4/current/BP-1938682392-172.31.14.131-1689542128252] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-16 21:16:17,067 WARN [Listener at localhost/43605] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-16 21:16:17,071 INFO [Listener at localhost/43605] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-16 21:16:17,177 WARN [BP-1938682392-172.31.14.131-1689542128252 heartbeating to localhost/127.0.0.1:35261] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-16 21:16:17,177 WARN [BP-1938682392-172.31.14.131-1689542128252 heartbeating to localhost/127.0.0.1:35261] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1938682392-172.31.14.131-1689542128252 (Datanode Uuid 6e68cdb2-c990-4136-ad44-2fec182ce26b) service to localhost/127.0.0.1:35261 2023-07-16 21:16:17,178 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5/dfs/data/data1/current/BP-1938682392-172.31.14.131-1689542128252] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-16 21:16:17,179 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/26d2aaeb-0d45-58d1-aeb8-aad5783be511/cluster_e02c4075-9296-82cb-39f6-1beae97724e5/dfs/data/data2/current/BP-1938682392-172.31.14.131-1689542128252] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-16 21:16:17,213 INFO [Listener at localhost/43605] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-16 21:16:17,342 INFO [Listener at localhost/43605] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-16 21:16:17,420 INFO [Listener at localhost/43605] hbase.HBaseTestingUtility(1293): Minicluster is down