2023-02-13 05:01:10,774 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2 2023-02-13 05:01:10,787 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2 timeout: 13 mins 2023-02-13 05:01:10,820 INFO [Time-limited test] hbase.ResourceChecker(147): before: client.TestAsyncClusterAdminApi2#testStop Thread=8, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=150, ProcessCount=168, AvailableMemoryMB=4203 2023-02-13 05:01:10,826 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-13 05:01:10,827 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/cluster_f3f6ca08-de39-71f7-ec28-b21a04d3bd2a, deleteOnExit=true 2023-02-13 05:01:10,827 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-13 05:01:10,827 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/test.cache.data in system properties and HBase conf 2023-02-13 05:01:10,828 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/hadoop.tmp.dir in system properties and HBase conf 2023-02-13 05:01:10,828 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/hadoop.log.dir in system properties and HBase conf 2023-02-13 05:01:10,829 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-13 05:01:10,829 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-13 05:01:10,829 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-13 05:01:10,944 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-13 05:01:11,307 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-13 05:01:11,310 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-13 05:01:11,311 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-13 05:01:11,311 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-13 05:01:11,311 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 05:01:11,311 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-13 05:01:11,312 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-13 05:01:11,312 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 05:01:11,312 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 05:01:11,313 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-13 05:01:11,313 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/nfs.dump.dir in system properties and HBase conf 2023-02-13 05:01:11,313 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/java.io.tmpdir in system properties and HBase conf 2023-02-13 05:01:11,313 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 05:01:11,314 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-13 05:01:11,314 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-13 05:01:11,768 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 05:01:11,774 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 05:01:12,047 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-13 05:01:12,201 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-13 05:01:12,215 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 05:01:12,253 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 05:01:12,284 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/java.io.tmpdir/Jetty_localhost_36947_hdfs____.1b2lgd/webapp 2023-02-13 05:01:12,448 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36947 2023-02-13 05:01:12,458 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 05:01:12,458 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 05:01:12,876 WARN [Listener at localhost/43091] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 05:01:12,943 WARN [Listener at localhost/43091] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 05:01:12,961 WARN [Listener at localhost/43091] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 05:01:12,967 INFO [Listener at localhost/43091] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 05:01:12,971 INFO [Listener at localhost/43091] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/java.io.tmpdir/Jetty_localhost_39471_datanode____m5cpfz/webapp 2023-02-13 05:01:13,066 INFO [Listener at localhost/43091] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39471 2023-02-13 05:01:13,374 WARN [Listener at localhost/38025] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 05:01:13,386 WARN [Listener at localhost/38025] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 05:01:13,390 WARN [Listener at localhost/38025] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 05:01:13,392 INFO [Listener at localhost/38025] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 05:01:13,401 INFO [Listener at localhost/38025] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/java.io.tmpdir/Jetty_localhost_38141_datanode____.j4dcj4/webapp 2023-02-13 05:01:13,505 INFO [Listener at localhost/38025] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38141 2023-02-13 05:01:13,514 WARN [Listener at localhost/45965] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 05:01:13,527 WARN [Listener at localhost/45965] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 05:01:13,530 WARN [Listener at localhost/45965] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 05:01:13,532 INFO [Listener at localhost/45965] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 05:01:13,539 INFO [Listener at localhost/45965] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/java.io.tmpdir/Jetty_localhost_40941_datanode____.duyr33/webapp 2023-02-13 05:01:13,671 INFO [Listener at localhost/45965] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40941 2023-02-13 05:01:13,679 WARN [Listener at localhost/37103] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 05:01:13,849 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xdbf9ac3fa03725bf: Processing first storage report for DS-a2ed32cc-b868-45a2-8f65-af0fdd67d399 from datanode 4d409657-b597-4a45-bf1d-41673fe1d75c 2023-02-13 05:01:13,850 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xdbf9ac3fa03725bf: from storage DS-a2ed32cc-b868-45a2-8f65-af0fdd67d399 node DatanodeRegistration(127.0.0.1:42027, datanodeUuid=4d409657-b597-4a45-bf1d-41673fe1d75c, infoPort=40705, infoSecurePort=0, ipcPort=37103, storageInfo=lv=-57;cid=testClusterID;nsid=1569380875;c=1676264471844), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-13 05:01:13,851 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x9cf9e605f4b79739: Processing first storage report for DS-1334d928-92cb-44e0-9c1a-14ade65a5b51 from datanode 44841879-a436-44ba-8bbb-055c9c1a883f 2023-02-13 05:01:13,851 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x9cf9e605f4b79739: from storage DS-1334d928-92cb-44e0-9c1a-14ade65a5b51 node DatanodeRegistration(127.0.0.1:36993, datanodeUuid=44841879-a436-44ba-8bbb-055c9c1a883f, infoPort=33747, infoSecurePort=0, ipcPort=38025, storageInfo=lv=-57;cid=testClusterID;nsid=1569380875;c=1676264471844), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 05:01:13,851 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x571b39c2e9c5ce67: Processing first storage report for DS-39a692cd-d6bf-4743-8d08-ed973eb7a0df from datanode 255ddfb9-4f75-4ad5-b387-eee2121ba847 2023-02-13 05:01:13,851 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x571b39c2e9c5ce67: from storage DS-39a692cd-d6bf-4743-8d08-ed973eb7a0df node DatanodeRegistration(127.0.0.1:43975, datanodeUuid=255ddfb9-4f75-4ad5-b387-eee2121ba847, infoPort=39649, infoSecurePort=0, ipcPort=45965, storageInfo=lv=-57;cid=testClusterID;nsid=1569380875;c=1676264471844), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 05:01:13,851 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xdbf9ac3fa03725bf: Processing first storage report for DS-98341472-3652-4221-8c3e-87933fba0f69 from datanode 4d409657-b597-4a45-bf1d-41673fe1d75c 2023-02-13 05:01:13,851 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xdbf9ac3fa03725bf: from storage DS-98341472-3652-4221-8c3e-87933fba0f69 node DatanodeRegistration(127.0.0.1:42027, datanodeUuid=4d409657-b597-4a45-bf1d-41673fe1d75c, infoPort=40705, infoSecurePort=0, ipcPort=37103, storageInfo=lv=-57;cid=testClusterID;nsid=1569380875;c=1676264471844), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 05:01:13,851 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x9cf9e605f4b79739: Processing first storage report for DS-c1441381-60d9-4bba-a14e-0d36af8ee0a2 from datanode 44841879-a436-44ba-8bbb-055c9c1a883f 2023-02-13 05:01:13,851 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x9cf9e605f4b79739: from storage DS-c1441381-60d9-4bba-a14e-0d36af8ee0a2 node DatanodeRegistration(127.0.0.1:36993, datanodeUuid=44841879-a436-44ba-8bbb-055c9c1a883f, infoPort=33747, infoSecurePort=0, ipcPort=38025, storageInfo=lv=-57;cid=testClusterID;nsid=1569380875;c=1676264471844), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 05:01:13,852 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x571b39c2e9c5ce67: Processing first storage report for DS-10e6a529-234b-4241-9fd2-ce344c2faaa0 from datanode 255ddfb9-4f75-4ad5-b387-eee2121ba847 2023-02-13 05:01:13,852 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x571b39c2e9c5ce67: from storage DS-10e6a529-234b-4241-9fd2-ce344c2faaa0 node DatanodeRegistration(127.0.0.1:43975, datanodeUuid=255ddfb9-4f75-4ad5-b387-eee2121ba847, infoPort=39649, infoSecurePort=0, ipcPort=45965, storageInfo=lv=-57;cid=testClusterID;nsid=1569380875;c=1676264471844), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 05:01:14,069 DEBUG [Listener at localhost/37103] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2 2023-02-13 05:01:14,129 INFO [Listener at localhost/37103] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/cluster_f3f6ca08-de39-71f7-ec28-b21a04d3bd2a/zookeeper_0, clientPort=60873, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/cluster_f3f6ca08-de39-71f7-ec28-b21a04d3bd2a/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/cluster_f3f6ca08-de39-71f7-ec28-b21a04d3bd2a/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-13 05:01:14,142 INFO [Listener at localhost/37103] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=60873 2023-02-13 05:01:14,152 INFO [Listener at localhost/37103] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:14,155 INFO [Listener at localhost/37103] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:14,810 INFO [Listener at localhost/37103] util.FSUtils(479): Created version file at hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22 with version=8 2023-02-13 05:01:14,810 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/hbase-staging 2023-02-13 05:01:15,107 INFO [Listener at localhost/37103] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-13 05:01:15,550 INFO [Listener at localhost/37103] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=6 2023-02-13 05:01:15,582 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:15,582 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:15,582 INFO [Listener at localhost/37103] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 05:01:15,583 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:15,583 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 05:01:15,727 INFO [Listener at localhost/37103] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 05:01:15,803 DEBUG [Listener at localhost/37103] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-13 05:01:15,896 INFO [Listener at localhost/37103] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38439 2023-02-13 05:01:15,905 INFO [Listener at localhost/37103] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:15,907 INFO [Listener at localhost/37103] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:15,927 INFO [Listener at localhost/37103] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38439 connecting to ZooKeeper ensemble=127.0.0.1:60873 2023-02-13 05:01:15,965 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:384390x0, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 05:01:15,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:38439-0x1004c0defc80000 connected 2023-02-13 05:01:15,992 DEBUG [Listener at localhost/37103] zookeeper.ZKUtil(164): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 05:01:15,992 DEBUG [Listener at localhost/37103] zookeeper.ZKUtil(164): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 05:01:15,996 DEBUG [Listener at localhost/37103] zookeeper.ZKUtil(164): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 05:01:16,003 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38439 2023-02-13 05:01:16,003 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38439 2023-02-13 05:01:16,004 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38439 2023-02-13 05:01:16,004 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38439 2023-02-13 05:01:16,004 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38439 2023-02-13 05:01:16,010 INFO [Listener at localhost/37103] master.HMaster(439): hbase.rootdir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22, hbase.cluster.distributed=false 2023-02-13 05:01:16,080 INFO [Listener at localhost/37103] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-13 05:01:16,080 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:16,080 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:16,080 INFO [Listener at localhost/37103] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 05:01:16,080 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:16,080 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 05:01:16,085 INFO [Listener at localhost/37103] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 05:01:16,089 INFO [Listener at localhost/37103] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37253 2023-02-13 05:01:16,091 INFO [Listener at localhost/37103] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 05:01:16,097 DEBUG [Listener at localhost/37103] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 05:01:16,098 INFO [Listener at localhost/37103] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:16,100 INFO [Listener at localhost/37103] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:16,102 INFO [Listener at localhost/37103] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37253 connecting to ZooKeeper ensemble=127.0.0.1:60873 2023-02-13 05:01:16,107 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:372530x0, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 05:01:16,108 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:37253-0x1004c0defc80001 connected 2023-02-13 05:01:16,109 DEBUG [Listener at localhost/37103] zookeeper.ZKUtil(164): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 05:01:16,110 DEBUG [Listener at localhost/37103] zookeeper.ZKUtil(164): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 05:01:16,111 DEBUG [Listener at localhost/37103] zookeeper.ZKUtil(164): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 05:01:16,111 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37253 2023-02-13 05:01:16,112 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37253 2023-02-13 05:01:16,112 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37253 2023-02-13 05:01:16,113 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37253 2023-02-13 05:01:16,113 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37253 2023-02-13 05:01:16,126 INFO [Listener at localhost/37103] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-13 05:01:16,126 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:16,127 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:16,127 INFO [Listener at localhost/37103] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 05:01:16,128 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:16,128 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 05:01:16,128 INFO [Listener at localhost/37103] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 05:01:16,130 INFO [Listener at localhost/37103] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42931 2023-02-13 05:01:16,130 INFO [Listener at localhost/37103] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 05:01:16,132 DEBUG [Listener at localhost/37103] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 05:01:16,133 INFO [Listener at localhost/37103] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:16,135 INFO [Listener at localhost/37103] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:16,138 INFO [Listener at localhost/37103] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42931 connecting to ZooKeeper ensemble=127.0.0.1:60873 2023-02-13 05:01:16,142 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:429310x0, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 05:01:16,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:42931-0x1004c0defc80002 connected 2023-02-13 05:01:16,143 DEBUG [Listener at localhost/37103] zookeeper.ZKUtil(164): regionserver:42931-0x1004c0defc80002, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 05:01:16,144 DEBUG [Listener at localhost/37103] zookeeper.ZKUtil(164): regionserver:42931-0x1004c0defc80002, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 05:01:16,145 DEBUG [Listener at localhost/37103] zookeeper.ZKUtil(164): regionserver:42931-0x1004c0defc80002, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 05:01:16,145 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42931 2023-02-13 05:01:16,146 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42931 2023-02-13 05:01:16,147 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42931 2023-02-13 05:01:16,147 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42931 2023-02-13 05:01:16,147 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42931 2023-02-13 05:01:16,160 INFO [Listener at localhost/37103] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-13 05:01:16,160 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:16,160 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:16,160 INFO [Listener at localhost/37103] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 05:01:16,160 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:16,160 INFO [Listener at localhost/37103] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 05:01:16,160 INFO [Listener at localhost/37103] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 05:01:16,162 INFO [Listener at localhost/37103] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33715 2023-02-13 05:01:16,163 INFO [Listener at localhost/37103] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 05:01:16,164 DEBUG [Listener at localhost/37103] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 05:01:16,165 INFO [Listener at localhost/37103] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:16,167 INFO [Listener at localhost/37103] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:16,169 INFO [Listener at localhost/37103] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33715 connecting to ZooKeeper ensemble=127.0.0.1:60873 2023-02-13 05:01:16,172 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:337150x0, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 05:01:16,173 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:33715-0x1004c0defc80003 connected 2023-02-13 05:01:16,173 DEBUG [Listener at localhost/37103] zookeeper.ZKUtil(164): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 05:01:16,174 DEBUG [Listener at localhost/37103] zookeeper.ZKUtil(164): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 05:01:16,175 DEBUG [Listener at localhost/37103] zookeeper.ZKUtil(164): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 05:01:16,176 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33715 2023-02-13 05:01:16,176 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33715 2023-02-13 05:01:16,177 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33715 2023-02-13 05:01:16,177 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33715 2023-02-13 05:01:16,177 DEBUG [Listener at localhost/37103] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33715 2023-02-13 05:01:16,179 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,38439,1676264474950 2023-02-13 05:01:16,187 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 05:01:16,189 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,38439,1676264474950 2023-02-13 05:01:16,206 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 05:01:16,206 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:42931-0x1004c0defc80002, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 05:01:16,206 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 05:01:16,206 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 05:01:16,206 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:16,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 05:01:16,209 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,38439,1676264474950 from backup master directory 2023-02-13 05:01:16,209 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 05:01:16,213 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,38439,1676264474950 2023-02-13 05:01:16,213 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 05:01:16,214 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 05:01:16,214 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,38439,1676264474950 2023-02-13 05:01:16,216 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-13 05:01:16,218 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-13 05:01:16,305 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/hbase.id with ID: 51c5eaa3-e5ad-4477-8547-6b4cf6fbcaa9 2023-02-13 05:01:16,345 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:16,359 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:16,407 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x6b26e56d to 127.0.0.1:60873 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 05:01:16,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@78988530, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 05:01:16,460 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 05:01:16,462 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 05:01:16,479 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-13 05:01:16,479 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-13 05:01:16,481 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 05:01:16,484 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 05:01:16,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 05:01:16,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/MasterData/data/master/store-tmp 2023-02-13 05:01:16,581 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 05:01:16,581 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 05:01:16,581 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:16,581 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:16,582 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 05:01:16,582 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:16,582 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:16,582 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676264476581Disabling compacts and flushes for region at 1676264476581Disabling writes for close at 1676264476582 (+1 ms)Writing region close event to WAL at 1676264476582Closed at 1676264476582 2023-02-13 05:01:16,583 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/MasterData/WALs/jenkins-hbase4.apache.org,38439,1676264474950 2023-02-13 05:01:16,603 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38439%2C1676264474950, suffix=, logDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/MasterData/WALs/jenkins-hbase4.apache.org,38439,1676264474950, archiveDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/MasterData/oldWALs, maxLogs=10 2023-02-13 05:01:16,665 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36993,DS-1334d928-92cb-44e0-9c1a-14ade65a5b51,DISK] 2023-02-13 05:01:16,665 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42027,DS-a2ed32cc-b868-45a2-8f65-af0fdd67d399,DISK] 2023-02-13 05:01:16,665 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43975,DS-39a692cd-d6bf-4743-8d08-ed973eb7a0df,DISK] 2023-02-13 05:01:16,673 DEBUG [RS-EventLoopGroup-5-1] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 05:01:16,745 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/MasterData/WALs/jenkins-hbase4.apache.org,38439,1676264474950/jenkins-hbase4.apache.org%2C38439%2C1676264474950.1676264476613 2023-02-13 05:01:16,746 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43975,DS-39a692cd-d6bf-4743-8d08-ed973eb7a0df,DISK], DatanodeInfoWithStorage[127.0.0.1:42027,DS-a2ed32cc-b868-45a2-8f65-af0fdd67d399,DISK], DatanodeInfoWithStorage[127.0.0.1:36993,DS-1334d928-92cb-44e0-9c1a-14ade65a5b51,DISK]] 2023-02-13 05:01:16,746 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 05:01:16,747 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 05:01:16,750 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 05:01:16,751 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 05:01:16,806 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 05:01:16,813 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 05:01:16,836 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 05:01:16,847 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:16,853 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 05:01:16,854 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 05:01:16,866 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 05:01:16,870 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 05:01:16,871 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63474208, jitterRate=-0.05416059494018555}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 05:01:16,871 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676264476775Initializing all the Stores at 1676264476777 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264476778 (+1 ms)Cleaning up temporary data from old regions at 1676264476859 (+81 ms)Cleaning up detritus from prior splits at 1676264476860 (+1 ms)Region opened successfully at 1676264476871 (+11 ms) 2023-02-13 05:01:16,872 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 05:01:16,890 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-13 05:01:16,890 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-13 05:01:16,893 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 05:01:16,894 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 05:01:16,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 30 msec 2023-02-13 05:01:16,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 05:01:16,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-13 05:01:16,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-13 05:01:16,980 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 05:01:16,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 05:01:16,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-13 05:01:16,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 05:01:16,993 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 05:01:16,997 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:16,998 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 05:01:16,999 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 05:01:17,011 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 05:01:17,015 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 05:01:17,015 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:42931-0x1004c0defc80002, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 05:01:17,015 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 05:01:17,015 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 05:01:17,016 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:17,016 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,38439,1676264474950, sessionid=0x1004c0defc80000, setting cluster-up flag (Was=false) 2023-02-13 05:01:17,030 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:17,041 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 05:01:17,043 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38439,1676264474950 2023-02-13 05:01:17,048 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:17,052 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 05:01:17,054 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38439,1676264474950 2023-02-13 05:01:17,056 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/.hbase-snapshot/.tmp 2023-02-13 05:01:17,081 INFO [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(952): ClusterId : 51c5eaa3-e5ad-4477-8547-6b4cf6fbcaa9 2023-02-13 05:01:17,081 INFO [RS:1;jenkins-hbase4:42931] regionserver.HRegionServer(952): ClusterId : 51c5eaa3-e5ad-4477-8547-6b4cf6fbcaa9 2023-02-13 05:01:17,081 INFO [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(952): ClusterId : 51c5eaa3-e5ad-4477-8547-6b4cf6fbcaa9 2023-02-13 05:01:17,088 DEBUG [RS:0;jenkins-hbase4:37253] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 05:01:17,088 DEBUG [RS:2;jenkins-hbase4:33715] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 05:01:17,088 DEBUG [RS:1;jenkins-hbase4:42931] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 05:01:17,095 DEBUG [RS:2;jenkins-hbase4:33715] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 05:01:17,095 DEBUG [RS:0;jenkins-hbase4:37253] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 05:01:17,095 DEBUG [RS:1;jenkins-hbase4:42931] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 05:01:17,095 DEBUG [RS:0;jenkins-hbase4:37253] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 05:01:17,095 DEBUG [RS:2;jenkins-hbase4:33715] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 05:01:17,096 DEBUG [RS:1;jenkins-hbase4:42931] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 05:01:17,100 DEBUG [RS:0;jenkins-hbase4:37253] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 05:01:17,100 DEBUG [RS:2;jenkins-hbase4:33715] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 05:01:17,100 DEBUG [RS:1;jenkins-hbase4:42931] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 05:01:17,103 DEBUG [RS:2;jenkins-hbase4:33715] zookeeper.ReadOnlyZKClient(139): Connect 0x71b88deb to 127.0.0.1:60873 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 05:01:17,103 DEBUG [RS:0;jenkins-hbase4:37253] zookeeper.ReadOnlyZKClient(139): Connect 0x2a3b18e1 to 127.0.0.1:60873 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 05:01:17,103 DEBUG [RS:1;jenkins-hbase4:42931] zookeeper.ReadOnlyZKClient(139): Connect 0x03dd11f3 to 127.0.0.1:60873 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 05:01:17,117 DEBUG [RS:0;jenkins-hbase4:37253] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@24fce3d6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 05:01:17,117 DEBUG [RS:2;jenkins-hbase4:33715] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7f7f6dac, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 05:01:17,118 DEBUG [RS:1;jenkins-hbase4:42931] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@d1afff8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 05:01:17,118 DEBUG [RS:0;jenkins-hbase4:37253] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@20c37391, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 05:01:17,118 DEBUG [RS:2;jenkins-hbase4:33715] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7b40e69b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 05:01:17,118 DEBUG [RS:1;jenkins-hbase4:42931] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@9257a83, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 05:01:17,141 DEBUG [RS:1;jenkins-hbase4:42931] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:42931 2023-02-13 05:01:17,144 INFO [RS:1;jenkins-hbase4:42931] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 05:01:17,144 DEBUG [RS:2;jenkins-hbase4:33715] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:33715 2023-02-13 05:01:17,145 INFO [RS:1;jenkins-hbase4:42931] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 05:01:17,145 INFO [RS:2;jenkins-hbase4:33715] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 05:01:17,145 DEBUG [RS:1;jenkins-hbase4:42931] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 05:01:17,146 INFO [RS:2;jenkins-hbase4:33715] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 05:01:17,146 DEBUG [RS:0;jenkins-hbase4:37253] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:37253 2023-02-13 05:01:17,146 DEBUG [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 05:01:17,146 INFO [RS:0;jenkins-hbase4:37253] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 05:01:17,146 INFO [RS:0;jenkins-hbase4:37253] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 05:01:17,146 DEBUG [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 05:01:17,149 INFO [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,38439,1676264474950 with isa=jenkins-hbase4.apache.org/172.31.14.131:37253, startcode=1676264476079 2023-02-13 05:01:17,149 INFO [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,38439,1676264474950 with isa=jenkins-hbase4.apache.org/172.31.14.131:33715, startcode=1676264476159 2023-02-13 05:01:17,149 INFO [RS:1;jenkins-hbase4:42931] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,38439,1676264474950 with isa=jenkins-hbase4.apache.org/172.31.14.131:42931, startcode=1676264476125 2023-02-13 05:01:17,168 DEBUG [RS:2;jenkins-hbase4:33715] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 05:01:17,168 DEBUG [RS:1;jenkins-hbase4:42931] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 05:01:17,168 DEBUG [RS:0;jenkins-hbase4:37253] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 05:01:17,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-13 05:01:17,206 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 05:01:17,206 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 05:01:17,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 05:01:17,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 05:01:17,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-13 05:01:17,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 05:01:17,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,208 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264507208 2023-02-13 05:01:17,211 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 05:01:17,214 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 05:01:17,215 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-13 05:01:17,221 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 05:01:17,223 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 05:01:17,224 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49171, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 05:01:17,224 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48279, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 05:01:17,224 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49173, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 05:01:17,233 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 05:01:17,233 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 05:01:17,234 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 05:01:17,234 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 05:01:17,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38439] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 05:01:17,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38439] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 05:01:17,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38439] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 05:01:17,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 05:01:17,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 05:01:17,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 05:01:17,243 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 05:01:17,243 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 05:01:17,247 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676264477247,5,FailOnTimeoutGroup] 2023-02-13 05:01:17,247 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676264477247,5,FailOnTimeoutGroup] 2023-02-13 05:01:17,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 05:01:17,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,250 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,272 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 05:01:17,273 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 05:01:17,273 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22 2023-02-13 05:01:17,275 DEBUG [RS:1;jenkins-hbase4:42931] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 05:01:17,275 DEBUG [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 05:01:17,275 DEBUG [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 05:01:17,276 WARN [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-13 05:01:17,276 WARN [RS:1;jenkins-hbase4:42931] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-13 05:01:17,276 WARN [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-13 05:01:17,295 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 05:01:17,297 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 05:01:17,299 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/info 2023-02-13 05:01:17,300 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 05:01:17,301 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:17,301 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 05:01:17,303 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/rep_barrier 2023-02-13 05:01:17,304 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 05:01:17,305 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:17,305 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 05:01:17,307 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/table 2023-02-13 05:01:17,307 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 05:01:17,308 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:17,309 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740 2023-02-13 05:01:17,310 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740 2023-02-13 05:01:17,313 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 05:01:17,315 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 05:01:17,318 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 05:01:17,319 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62738383, jitterRate=-0.0651252418756485}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 05:01:17,319 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676264477295Initializing all the Stores at 1676264477297 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676264477297Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264477297Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676264477297Cleaning up temporary data from old regions at 1676264477312 (+15 ms)Cleaning up detritus from prior splits at 1676264477312Region opened successfully at 1676264477319 (+7 ms) 2023-02-13 05:01:17,319 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 05:01:17,319 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 05:01:17,319 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 05:01:17,320 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 05:01:17,320 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 05:01:17,320 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 05:01:17,321 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676264477319Disabling compacts and flushes for region at 1676264477319Disabling writes for close at 1676264477320 (+1 ms)Writing region close event to WAL at 1676264477320Closed at 1676264477320 2023-02-13 05:01:17,325 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 05:01:17,325 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-13 05:01:17,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 05:01:17,346 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 05:01:17,350 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 05:01:17,377 INFO [RS:1;jenkins-hbase4:42931] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,38439,1676264474950 with isa=jenkins-hbase4.apache.org/172.31.14.131:42931, startcode=1676264476125 2023-02-13 05:01:17,377 INFO [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,38439,1676264474950 with isa=jenkins-hbase4.apache.org/172.31.14.131:33715, startcode=1676264476159 2023-02-13 05:01:17,377 INFO [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,38439,1676264474950 with isa=jenkins-hbase4.apache.org/172.31.14.131:37253, startcode=1676264476079 2023-02-13 05:01:17,384 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38439] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33715,1676264476159 2023-02-13 05:01:17,384 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38439] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42931,1676264476125 2023-02-13 05:01:17,385 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38439] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,37253,1676264476079 2023-02-13 05:01:17,391 DEBUG [RS:1;jenkins-hbase4:42931] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22 2023-02-13 05:01:17,391 DEBUG [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22 2023-02-13 05:01:17,391 DEBUG [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22 2023-02-13 05:01:17,391 DEBUG [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:43091 2023-02-13 05:01:17,391 DEBUG [RS:1;jenkins-hbase4:42931] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:43091 2023-02-13 05:01:17,391 DEBUG [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 05:01:17,391 DEBUG [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:43091 2023-02-13 05:01:17,391 DEBUG [RS:1;jenkins-hbase4:42931] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 05:01:17,392 DEBUG [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 05:01:17,399 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 05:01:17,400 DEBUG [RS:1;jenkins-hbase4:42931] zookeeper.ZKUtil(162): regionserver:42931-0x1004c0defc80002, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42931,1676264476125 2023-02-13 05:01:17,400 DEBUG [RS:0;jenkins-hbase4:37253] zookeeper.ZKUtil(162): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37253,1676264476079 2023-02-13 05:01:17,400 WARN [RS:1;jenkins-hbase4:42931] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 05:01:17,400 DEBUG [RS:2;jenkins-hbase4:33715] zookeeper.ZKUtil(162): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33715,1676264476159 2023-02-13 05:01:17,401 INFO [RS:1;jenkins-hbase4:42931] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 05:01:17,400 WARN [RS:0;jenkins-hbase4:37253] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 05:01:17,401 WARN [RS:2;jenkins-hbase4:33715] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 05:01:17,401 DEBUG [RS:1;jenkins-hbase4:42931] regionserver.HRegionServer(1947): logDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/WALs/jenkins-hbase4.apache.org,42931,1676264476125 2023-02-13 05:01:17,402 INFO [RS:2;jenkins-hbase4:33715] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 05:01:17,401 INFO [RS:0;jenkins-hbase4:37253] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 05:01:17,402 DEBUG [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(1947): logDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/WALs/jenkins-hbase4.apache.org,33715,1676264476159 2023-02-13 05:01:17,402 DEBUG [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(1947): logDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/WALs/jenkins-hbase4.apache.org,37253,1676264476079 2023-02-13 05:01:17,402 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,37253,1676264476079] 2023-02-13 05:01:17,403 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33715,1676264476159] 2023-02-13 05:01:17,403 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42931,1676264476125] 2023-02-13 05:01:17,415 DEBUG [RS:1;jenkins-hbase4:42931] zookeeper.ZKUtil(162): regionserver:42931-0x1004c0defc80002, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42931,1676264476125 2023-02-13 05:01:17,416 DEBUG [RS:2;jenkins-hbase4:33715] zookeeper.ZKUtil(162): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42931,1676264476125 2023-02-13 05:01:17,416 DEBUG [RS:0;jenkins-hbase4:37253] zookeeper.ZKUtil(162): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42931,1676264476125 2023-02-13 05:01:17,416 DEBUG [RS:1;jenkins-hbase4:42931] zookeeper.ZKUtil(162): regionserver:42931-0x1004c0defc80002, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37253,1676264476079 2023-02-13 05:01:17,416 DEBUG [RS:2;jenkins-hbase4:33715] zookeeper.ZKUtil(162): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37253,1676264476079 2023-02-13 05:01:17,416 DEBUG [RS:0;jenkins-hbase4:37253] zookeeper.ZKUtil(162): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37253,1676264476079 2023-02-13 05:01:17,416 DEBUG [RS:1;jenkins-hbase4:42931] zookeeper.ZKUtil(162): regionserver:42931-0x1004c0defc80002, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33715,1676264476159 2023-02-13 05:01:17,417 DEBUG [RS:2;jenkins-hbase4:33715] zookeeper.ZKUtil(162): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33715,1676264476159 2023-02-13 05:01:17,417 DEBUG [RS:0;jenkins-hbase4:37253] zookeeper.ZKUtil(162): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33715,1676264476159 2023-02-13 05:01:17,426 DEBUG [RS:1;jenkins-hbase4:42931] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 05:01:17,426 DEBUG [RS:2;jenkins-hbase4:33715] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 05:01:17,426 DEBUG [RS:0;jenkins-hbase4:37253] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 05:01:17,435 INFO [RS:2;jenkins-hbase4:33715] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 05:01:17,435 INFO [RS:1;jenkins-hbase4:42931] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 05:01:17,435 INFO [RS:0;jenkins-hbase4:37253] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 05:01:17,455 INFO [RS:1;jenkins-hbase4:42931] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 05:01:17,455 INFO [RS:2;jenkins-hbase4:33715] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 05:01:17,455 INFO [RS:0;jenkins-hbase4:37253] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 05:01:17,460 INFO [RS:1;jenkins-hbase4:42931] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 05:01:17,460 INFO [RS:0;jenkins-hbase4:37253] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 05:01:17,460 INFO [RS:2;jenkins-hbase4:33715] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 05:01:17,460 INFO [RS:0;jenkins-hbase4:37253] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,460 INFO [RS:1;jenkins-hbase4:42931] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,460 INFO [RS:2;jenkins-hbase4:33715] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,461 INFO [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 05:01:17,461 INFO [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 05:01:17,461 INFO [RS:1;jenkins-hbase4:42931] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 05:01:17,469 INFO [RS:1;jenkins-hbase4:42931] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,469 INFO [RS:0;jenkins-hbase4:37253] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,469 INFO [RS:2;jenkins-hbase4:33715] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,469 DEBUG [RS:1;jenkins-hbase4:42931] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,469 DEBUG [RS:0;jenkins-hbase4:37253] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:1;jenkins-hbase4:42931] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:0;jenkins-hbase4:37253] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:1;jenkins-hbase4:42931] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:0;jenkins-hbase4:37253] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:1;jenkins-hbase4:42931] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:0;jenkins-hbase4:37253] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:1;jenkins-hbase4:42931] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:0;jenkins-hbase4:37253] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:1;jenkins-hbase4:42931] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 05:01:17,470 DEBUG [RS:0;jenkins-hbase4:37253] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 05:01:17,470 DEBUG [RS:1;jenkins-hbase4:42931] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:0;jenkins-hbase4:37253] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,469 DEBUG [RS:2;jenkins-hbase4:33715] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:0;jenkins-hbase4:37253] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:2;jenkins-hbase4:33715] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:0;jenkins-hbase4:37253] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:2;jenkins-hbase4:33715] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:0;jenkins-hbase4:37253] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,471 DEBUG [RS:2;jenkins-hbase4:33715] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,470 DEBUG [RS:1;jenkins-hbase4:42931] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,471 DEBUG [RS:2;jenkins-hbase4:33715] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,471 DEBUG [RS:1;jenkins-hbase4:42931] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,471 DEBUG [RS:2;jenkins-hbase4:33715] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 05:01:17,471 DEBUG [RS:1;jenkins-hbase4:42931] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,471 DEBUG [RS:2;jenkins-hbase4:33715] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,471 INFO [RS:0;jenkins-hbase4:37253] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,472 DEBUG [RS:2;jenkins-hbase4:33715] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,472 INFO [RS:0;jenkins-hbase4:37253] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,472 DEBUG [RS:2;jenkins-hbase4:33715] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,472 INFO [RS:0;jenkins-hbase4:37253] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,472 DEBUG [RS:2;jenkins-hbase4:33715] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:17,477 INFO [RS:1;jenkins-hbase4:42931] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,477 INFO [RS:1;jenkins-hbase4:42931] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,477 INFO [RS:2;jenkins-hbase4:33715] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,477 INFO [RS:1;jenkins-hbase4:42931] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,477 INFO [RS:2;jenkins-hbase4:33715] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,477 INFO [RS:2;jenkins-hbase4:33715] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,495 INFO [RS:1;jenkins-hbase4:42931] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 05:01:17,495 INFO [RS:2;jenkins-hbase4:33715] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 05:01:17,495 INFO [RS:0;jenkins-hbase4:37253] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 05:01:17,498 INFO [RS:2;jenkins-hbase4:33715] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33715,1676264476159-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,498 INFO [RS:1;jenkins-hbase4:42931] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42931,1676264476125-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,498 INFO [RS:0;jenkins-hbase4:37253] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37253,1676264476079-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,502 DEBUG [jenkins-hbase4:38439] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-13 05:01:17,507 DEBUG [jenkins-hbase4:38439] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 05:01:17,513 DEBUG [jenkins-hbase4:38439] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 05:01:17,513 DEBUG [jenkins-hbase4:38439] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 05:01:17,513 DEBUG [jenkins-hbase4:38439] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 05:01:17,513 DEBUG [jenkins-hbase4:38439] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 05:01:17,516 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33715,1676264476159, state=OPENING 2023-02-13 05:01:17,522 INFO [RS:1;jenkins-hbase4:42931] regionserver.Replication(203): jenkins-hbase4.apache.org,42931,1676264476125 started 2023-02-13 05:01:17,522 INFO [RS:1;jenkins-hbase4:42931] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,42931,1676264476125, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42931, sessionid=0x1004c0defc80002 2023-02-13 05:01:17,522 INFO [RS:0;jenkins-hbase4:37253] regionserver.Replication(203): jenkins-hbase4.apache.org,37253,1676264476079 started 2023-02-13 05:01:17,522 INFO [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,37253,1676264476079, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:37253, sessionid=0x1004c0defc80001 2023-02-13 05:01:17,523 INFO [RS:2;jenkins-hbase4:33715] regionserver.Replication(203): jenkins-hbase4.apache.org,33715,1676264476159 started 2023-02-13 05:01:17,523 DEBUG [RS:1;jenkins-hbase4:42931] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 05:01:17,523 DEBUG [RS:0;jenkins-hbase4:37253] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 05:01:17,523 DEBUG [RS:1;jenkins-hbase4:42931] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42931,1676264476125 2023-02-13 05:01:17,523 INFO [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,33715,1676264476159, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33715, sessionid=0x1004c0defc80003 2023-02-13 05:01:17,523 DEBUG [RS:1;jenkins-hbase4:42931] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42931,1676264476125' 2023-02-13 05:01:17,523 DEBUG [RS:0;jenkins-hbase4:37253] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,37253,1676264476079 2023-02-13 05:01:17,524 DEBUG [RS:1;jenkins-hbase4:42931] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 05:01:17,524 DEBUG [RS:2;jenkins-hbase4:33715] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 05:01:17,524 DEBUG [RS:0;jenkins-hbase4:37253] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37253,1676264476079' 2023-02-13 05:01:17,524 DEBUG [RS:0;jenkins-hbase4:37253] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 05:01:17,524 DEBUG [RS:2;jenkins-hbase4:33715] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33715,1676264476159 2023-02-13 05:01:17,524 DEBUG [RS:2;jenkins-hbase4:33715] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33715,1676264476159' 2023-02-13 05:01:17,525 DEBUG [RS:2;jenkins-hbase4:33715] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 05:01:17,524 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-13 05:01:17,525 DEBUG [RS:0;jenkins-hbase4:37253] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 05:01:17,525 DEBUG [RS:1;jenkins-hbase4:42931] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 05:01:17,525 DEBUG [RS:2;jenkins-hbase4:33715] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 05:01:17,526 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:17,527 DEBUG [RS:0;jenkins-hbase4:37253] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 05:01:17,527 DEBUG [RS:1;jenkins-hbase4:42931] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 05:01:17,527 DEBUG [RS:2;jenkins-hbase4:33715] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 05:01:17,527 DEBUG [RS:1;jenkins-hbase4:42931] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 05:01:17,527 DEBUG [RS:0;jenkins-hbase4:37253] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 05:01:17,527 DEBUG [RS:1;jenkins-hbase4:42931] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42931,1676264476125 2023-02-13 05:01:17,527 DEBUG [RS:2;jenkins-hbase4:33715] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 05:01:17,527 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 05:01:17,527 DEBUG [RS:1;jenkins-hbase4:42931] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42931,1676264476125' 2023-02-13 05:01:17,528 DEBUG [RS:1;jenkins-hbase4:42931] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 05:01:17,527 DEBUG [RS:0;jenkins-hbase4:37253] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,37253,1676264476079 2023-02-13 05:01:17,527 DEBUG [RS:2;jenkins-hbase4:33715] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33715,1676264476159 2023-02-13 05:01:17,528 DEBUG [RS:0;jenkins-hbase4:37253] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37253,1676264476079' 2023-02-13 05:01:17,528 DEBUG [RS:0;jenkins-hbase4:37253] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 05:01:17,528 DEBUG [RS:2;jenkins-hbase4:33715] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33715,1676264476159' 2023-02-13 05:01:17,528 DEBUG [RS:2;jenkins-hbase4:33715] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 05:01:17,528 DEBUG [RS:1;jenkins-hbase4:42931] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 05:01:17,529 DEBUG [RS:2;jenkins-hbase4:33715] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 05:01:17,529 DEBUG [RS:1;jenkins-hbase4:42931] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 05:01:17,529 DEBUG [RS:0;jenkins-hbase4:37253] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 05:01:17,529 INFO [RS:1;jenkins-hbase4:42931] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 05:01:17,530 INFO [RS:1;jenkins-hbase4:42931] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 05:01:17,530 DEBUG [RS:2;jenkins-hbase4:33715] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 05:01:17,530 INFO [RS:2;jenkins-hbase4:33715] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 05:01:17,530 DEBUG [RS:0;jenkins-hbase4:37253] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 05:01:17,531 INFO [RS:0;jenkins-hbase4:37253] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 05:01:17,530 INFO [RS:2;jenkins-hbase4:33715] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 05:01:17,532 INFO [RS:0;jenkins-hbase4:37253] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 05:01:17,533 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33715,1676264476159}] 2023-02-13 05:01:17,640 INFO [RS:1;jenkins-hbase4:42931] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42931%2C1676264476125, suffix=, logDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/WALs/jenkins-hbase4.apache.org,42931,1676264476125, archiveDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/oldWALs, maxLogs=32 2023-02-13 05:01:17,641 INFO [RS:0;jenkins-hbase4:37253] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37253%2C1676264476079, suffix=, logDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/WALs/jenkins-hbase4.apache.org,37253,1676264476079, archiveDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/oldWALs, maxLogs=32 2023-02-13 05:01:17,640 INFO [RS:2;jenkins-hbase4:33715] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33715%2C1676264476159, suffix=, logDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/WALs/jenkins-hbase4.apache.org,33715,1676264476159, archiveDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/oldWALs, maxLogs=32 2023-02-13 05:01:17,665 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36993,DS-1334d928-92cb-44e0-9c1a-14ade65a5b51,DISK] 2023-02-13 05:01:17,665 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42027,DS-a2ed32cc-b868-45a2-8f65-af0fdd67d399,DISK] 2023-02-13 05:01:17,666 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43975,DS-39a692cd-d6bf-4743-8d08-ed973eb7a0df,DISK] 2023-02-13 05:01:17,670 INFO [RS:0;jenkins-hbase4:37253] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/WALs/jenkins-hbase4.apache.org,37253,1676264476079/jenkins-hbase4.apache.org%2C37253%2C1676264476079.1676264477645 2023-02-13 05:01:17,673 DEBUG [RS:0;jenkins-hbase4:37253] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42027,DS-a2ed32cc-b868-45a2-8f65-af0fdd67d399,DISK], DatanodeInfoWithStorage[127.0.0.1:36993,DS-1334d928-92cb-44e0-9c1a-14ade65a5b51,DISK], DatanodeInfoWithStorage[127.0.0.1:43975,DS-39a692cd-d6bf-4743-8d08-ed973eb7a0df,DISK]] 2023-02-13 05:01:17,680 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42027,DS-a2ed32cc-b868-45a2-8f65-af0fdd67d399,DISK] 2023-02-13 05:01:17,681 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36993,DS-1334d928-92cb-44e0-9c1a-14ade65a5b51,DISK] 2023-02-13 05:01:17,684 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42027,DS-a2ed32cc-b868-45a2-8f65-af0fdd67d399,DISK] 2023-02-13 05:01:17,684 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43975,DS-39a692cd-d6bf-4743-8d08-ed973eb7a0df,DISK] 2023-02-13 05:01:17,684 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43975,DS-39a692cd-d6bf-4743-8d08-ed973eb7a0df,DISK] 2023-02-13 05:01:17,685 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36993,DS-1334d928-92cb-44e0-9c1a-14ade65a5b51,DISK] 2023-02-13 05:01:17,699 INFO [RS:1;jenkins-hbase4:42931] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/WALs/jenkins-hbase4.apache.org,42931,1676264476125/jenkins-hbase4.apache.org%2C42931%2C1676264476125.1676264477645 2023-02-13 05:01:17,699 INFO [RS:2;jenkins-hbase4:33715] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/WALs/jenkins-hbase4.apache.org,33715,1676264476159/jenkins-hbase4.apache.org%2C33715%2C1676264476159.1676264477645 2023-02-13 05:01:17,702 DEBUG [RS:1;jenkins-hbase4:42931] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36993,DS-1334d928-92cb-44e0-9c1a-14ade65a5b51,DISK], DatanodeInfoWithStorage[127.0.0.1:42027,DS-a2ed32cc-b868-45a2-8f65-af0fdd67d399,DISK], DatanodeInfoWithStorage[127.0.0.1:43975,DS-39a692cd-d6bf-4743-8d08-ed973eb7a0df,DISK]] 2023-02-13 05:01:17,702 DEBUG [RS:2;jenkins-hbase4:33715] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42027,DS-a2ed32cc-b868-45a2-8f65-af0fdd67d399,DISK], DatanodeInfoWithStorage[127.0.0.1:43975,DS-39a692cd-d6bf-4743-8d08-ed973eb7a0df,DISK], DatanodeInfoWithStorage[127.0.0.1:36993,DS-1334d928-92cb-44e0-9c1a-14ade65a5b51,DISK]] 2023-02-13 05:01:17,722 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33715,1676264476159 2023-02-13 05:01:17,725 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 05:01:17,727 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56088, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 05:01:17,739 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 05:01:17,739 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 05:01:17,743 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33715%2C1676264476159.meta, suffix=.meta, logDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/WALs/jenkins-hbase4.apache.org,33715,1676264476159, archiveDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/oldWALs, maxLogs=32 2023-02-13 05:01:17,763 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43975,DS-39a692cd-d6bf-4743-8d08-ed973eb7a0df,DISK] 2023-02-13 05:01:17,763 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36993,DS-1334d928-92cb-44e0-9c1a-14ade65a5b51,DISK] 2023-02-13 05:01:17,766 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42027,DS-a2ed32cc-b868-45a2-8f65-af0fdd67d399,DISK] 2023-02-13 05:01:17,774 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/WALs/jenkins-hbase4.apache.org,33715,1676264476159/jenkins-hbase4.apache.org%2C33715%2C1676264476159.meta.1676264477744.meta 2023-02-13 05:01:17,774 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43975,DS-39a692cd-d6bf-4743-8d08-ed973eb7a0df,DISK], DatanodeInfoWithStorage[127.0.0.1:36993,DS-1334d928-92cb-44e0-9c1a-14ade65a5b51,DISK], DatanodeInfoWithStorage[127.0.0.1:42027,DS-a2ed32cc-b868-45a2-8f65-af0fdd67d399,DISK]] 2023-02-13 05:01:17,774 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 05:01:17,776 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 05:01:17,794 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 05:01:17,799 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 05:01:17,804 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 05:01:17,804 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 05:01:17,804 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 05:01:17,804 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 05:01:17,807 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 05:01:17,809 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/info 2023-02-13 05:01:17,809 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/info 2023-02-13 05:01:17,809 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 05:01:17,812 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:17,812 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 05:01:17,815 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/rep_barrier 2023-02-13 05:01:17,815 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/rep_barrier 2023-02-13 05:01:17,816 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 05:01:17,817 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:17,817 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 05:01:17,819 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/table 2023-02-13 05:01:17,819 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/table 2023-02-13 05:01:17,819 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 05:01:17,820 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:17,822 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740 2023-02-13 05:01:17,825 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740 2023-02-13 05:01:17,829 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 05:01:17,831 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 05:01:17,834 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62111610, jitterRate=-0.07446488738059998}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 05:01:17,834 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676264477804Writing region info on filesystem at 1676264477805 (+1 ms)Initializing all the Stores at 1676264477806 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676264477806Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264477807 (+1 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676264477807Cleaning up temporary data from old regions at 1676264477827 (+20 ms)Cleaning up detritus from prior splits at 1676264477828 (+1 ms)Running coprocessor post-open hooks at 1676264477834 (+6 ms)Region opened successfully at 1676264477834 2023-02-13 05:01:17,845 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676264477715 2023-02-13 05:01:17,861 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 05:01:17,862 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 05:01:17,863 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33715,1676264476159, state=OPEN 2023-02-13 05:01:17,866 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 05:01:17,866 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 05:01:17,870 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-13 05:01:17,870 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33715,1676264476159 in 333 msec 2023-02-13 05:01:17,875 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-13 05:01:17,875 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 537 msec 2023-02-13 05:01:17,880 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 771 msec 2023-02-13 05:01:17,881 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676264476202, completionTime=-1 2023-02-13 05:01:17,881 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-13 05:01:17,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 05:01:17,947 DEBUG [hconnection-0x6bfa197-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 05:01:17,950 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56100, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 05:01:17,967 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 05:01:17,967 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264537967 2023-02-13 05:01:17,967 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264597967 2023-02-13 05:01:17,967 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 86 msec 2023-02-13 05:01:17,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38439,1676264474950-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38439,1676264474950-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38439,1676264474950-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,993 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38439, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:17,993 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:18,002 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-13 05:01:18,010 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-13 05:01:18,012 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 05:01:18,022 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-13 05:01:18,025 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 05:01:18,027 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 05:01:18,049 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/.tmp/data/hbase/namespace/00f7d0808277ee6962f65d76422c27a7 2023-02-13 05:01:18,052 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/.tmp/data/hbase/namespace/00f7d0808277ee6962f65d76422c27a7 empty. 2023-02-13 05:01:18,053 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/.tmp/data/hbase/namespace/00f7d0808277ee6962f65d76422c27a7 2023-02-13 05:01:18,053 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-13 05:01:18,089 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-13 05:01:18,092 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 00f7d0808277ee6962f65d76422c27a7, NAME => 'hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/.tmp 2023-02-13 05:01:18,110 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 05:01:18,111 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 00f7d0808277ee6962f65d76422c27a7, disabling compactions & flushes 2023-02-13 05:01:18,111 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7. 2023-02-13 05:01:18,111 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7. 2023-02-13 05:01:18,111 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7. after waiting 0 ms 2023-02-13 05:01:18,111 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7. 2023-02-13 05:01:18,111 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7. 2023-02-13 05:01:18,111 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 00f7d0808277ee6962f65d76422c27a7: Waiting for close lock at 1676264478111Disabling compacts and flushes for region at 1676264478111Disabling writes for close at 1676264478111Writing region close event to WAL at 1676264478111Closed at 1676264478111 2023-02-13 05:01:18,115 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 05:01:18,131 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676264478118"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264478118"}]},"ts":"1676264478118"} 2023-02-13 05:01:18,159 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 05:01:18,161 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 05:01:18,166 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264478161"}]},"ts":"1676264478161"} 2023-02-13 05:01:18,170 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-13 05:01:18,174 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 05:01:18,175 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 05:01:18,175 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 05:01:18,175 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 05:01:18,175 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 05:01:18,178 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=00f7d0808277ee6962f65d76422c27a7, ASSIGN}] 2023-02-13 05:01:18,180 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=00f7d0808277ee6962f65d76422c27a7, ASSIGN 2023-02-13 05:01:18,182 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=00f7d0808277ee6962f65d76422c27a7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37253,1676264476079; forceNewPlan=false, retain=false 2023-02-13 05:01:18,334 INFO [jenkins-hbase4:38439] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 05:01:18,335 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=00f7d0808277ee6962f65d76422c27a7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37253,1676264476079 2023-02-13 05:01:18,335 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676264478335"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264478335"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264478335"}]},"ts":"1676264478335"} 2023-02-13 05:01:18,339 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 00f7d0808277ee6962f65d76422c27a7, server=jenkins-hbase4.apache.org,37253,1676264476079}] 2023-02-13 05:01:18,494 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,37253,1676264476079 2023-02-13 05:01:18,495 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 05:01:18,499 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43322, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 05:01:18,505 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7. 2023-02-13 05:01:18,506 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 00f7d0808277ee6962f65d76422c27a7, NAME => 'hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7.', STARTKEY => '', ENDKEY => ''} 2023-02-13 05:01:18,507 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 00f7d0808277ee6962f65d76422c27a7 2023-02-13 05:01:18,507 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 05:01:18,507 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 00f7d0808277ee6962f65d76422c27a7 2023-02-13 05:01:18,507 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 00f7d0808277ee6962f65d76422c27a7 2023-02-13 05:01:18,509 INFO [StoreOpener-00f7d0808277ee6962f65d76422c27a7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 00f7d0808277ee6962f65d76422c27a7 2023-02-13 05:01:18,512 DEBUG [StoreOpener-00f7d0808277ee6962f65d76422c27a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/namespace/00f7d0808277ee6962f65d76422c27a7/info 2023-02-13 05:01:18,512 DEBUG [StoreOpener-00f7d0808277ee6962f65d76422c27a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/namespace/00f7d0808277ee6962f65d76422c27a7/info 2023-02-13 05:01:18,512 INFO [StoreOpener-00f7d0808277ee6962f65d76422c27a7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 00f7d0808277ee6962f65d76422c27a7 columnFamilyName info 2023-02-13 05:01:18,513 INFO [StoreOpener-00f7d0808277ee6962f65d76422c27a7-1] regionserver.HStore(310): Store=00f7d0808277ee6962f65d76422c27a7/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:18,515 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/namespace/00f7d0808277ee6962f65d76422c27a7 2023-02-13 05:01:18,515 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/namespace/00f7d0808277ee6962f65d76422c27a7 2023-02-13 05:01:18,519 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 00f7d0808277ee6962f65d76422c27a7 2023-02-13 05:01:18,523 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/namespace/00f7d0808277ee6962f65d76422c27a7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 05:01:18,524 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 00f7d0808277ee6962f65d76422c27a7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65121668, jitterRate=-0.029611527919769287}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 05:01:18,524 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 00f7d0808277ee6962f65d76422c27a7: Running coprocessor pre-open hook at 1676264478507Writing region info on filesystem at 1676264478507Initializing all the Stores at 1676264478509 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676264478509Cleaning up temporary data from old regions at 1676264478516 (+7 ms)Cleaning up detritus from prior splits at 1676264478517 (+1 ms)Running coprocessor post-open hooks at 1676264478524 (+7 ms)Region opened successfully at 1676264478524 2023-02-13 05:01:18,526 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7., pid=6, masterSystemTime=1676264478494 2023-02-13 05:01:18,530 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7. 2023-02-13 05:01:18,530 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7. 2023-02-13 05:01:18,532 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=00f7d0808277ee6962f65d76422c27a7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37253,1676264476079 2023-02-13 05:01:18,532 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676264478531"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264478531"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264478531"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264478531"}]},"ts":"1676264478531"} 2023-02-13 05:01:18,539 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-13 05:01:18,539 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 00f7d0808277ee6962f65d76422c27a7, server=jenkins-hbase4.apache.org,37253,1676264476079 in 196 msec 2023-02-13 05:01:18,543 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-13 05:01:18,543 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=00f7d0808277ee6962f65d76422c27a7, ASSIGN in 361 msec 2023-02-13 05:01:18,544 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 05:01:18,545 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264478544"}]},"ts":"1676264478544"} 2023-02-13 05:01:18,547 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-13 05:01:18,550 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 05:01:18,552 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 537 msec 2023-02-13 05:01:18,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-13 05:01:18,628 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-13 05:01:18,628 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:18,648 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 05:01:18,651 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43332, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 05:01:18,668 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-13 05:01:18,685 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 05:01:18,691 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 33 msec 2023-02-13 05:01:18,701 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-13 05:01:18,712 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 05:01:18,719 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 17 msec 2023-02-13 05:01:18,727 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 05:01:18,730 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 05:01:18,730 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.516sec 2023-02-13 05:01:18,733 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 05:01:18,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 05:01:18,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 05:01:18,735 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38439,1676264474950-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 05:01:18,737 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38439,1676264474950-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 05:01:18,747 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 05:01:18,787 DEBUG [Listener at localhost/37103] zookeeper.ReadOnlyZKClient(139): Connect 0x6d08d536 to 127.0.0.1:60873 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 05:01:18,792 DEBUG [Listener at localhost/37103] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@557f7f6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 05:01:18,805 DEBUG [hconnection-0x3f15301a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 05:01:18,817 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56110, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 05:01:18,827 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,38439,1676264474950 2023-02-13 05:01:18,827 DEBUG [Listener at localhost/37103] zookeeper.ReadOnlyZKClient(139): Connect 0x2700073d to 127.0.0.1:60873 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 05:01:18,837 DEBUG [ReadOnlyZKClient-127.0.0.1:60873@0x2700073d] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2fabc20d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 05:01:18,873 DEBUG [Listener at localhost/37103] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 05:01:18,876 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43336, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 05:01:18,877 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37253] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,37253,1676264476079' ***** 2023-02-13 05:01:18,877 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37253] regionserver.HRegionServer(2310): STOPPED: Called by admin client org.apache.hadoop.hbase.client.AsyncConnectionImpl@b9c4639 2023-02-13 05:01:18,878 INFO [RS:0;jenkins-hbase4:37253] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 05:01:18,879 INFO [RS:0;jenkins-hbase4:37253] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 05:01:18,879 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 05:01:18,879 INFO [RS:0;jenkins-hbase4:37253] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 05:01:18,880 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 05:01:18,881 INFO [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(3304): Received CLOSE for 00f7d0808277ee6962f65d76422c27a7 2023-02-13 05:01:18,881 INFO [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,37253,1676264476079 2023-02-13 05:01:18,882 DEBUG [RS:0;jenkins-hbase4:37253] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2a3b18e1 to 127.0.0.1:60873 2023-02-13 05:01:18,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 00f7d0808277ee6962f65d76422c27a7, disabling compactions & flushes 2023-02-13 05:01:18,882 DEBUG [RS:0;jenkins-hbase4:37253] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:18,882 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7. 2023-02-13 05:01:18,883 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7. 2023-02-13 05:01:18,883 INFO [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 05:01:18,883 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7. after waiting 0 ms 2023-02-13 05:01:18,883 DEBUG [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(1479): Online Regions={00f7d0808277ee6962f65d76422c27a7=hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7.} 2023-02-13 05:01:18,883 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7. 2023-02-13 05:01:18,884 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 00f7d0808277ee6962f65d76422c27a7 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-13 05:01:18,884 DEBUG [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(1505): Waiting on 00f7d0808277ee6962f65d76422c27a7 2023-02-13 05:01:18,885 DEBUG [Listener at localhost/37103] client.ConnectionUtils(586): Start fetching master stub from registry 2023-02-13 05:01:18,886 DEBUG [ReadOnlyZKClient-127.0.0.1:60873@0x2700073d] client.AsyncConnectionImpl(289): The fetched master address is jenkins-hbase4.apache.org,38439,1676264474950 2023-02-13 05:01:18,889 DEBUG [ReadOnlyZKClient-127.0.0.1:60873@0x2700073d] client.ConnectionUtils(594): The fetched master stub is org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$Stub@238a842 2023-02-13 05:01:18,894 DEBUG [ReadOnlyZKClient-127.0.0.1:60873@0x2700073d] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 05:01:18,897 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45282, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 05:01:18,898 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38439] master.MasterRpcServices(1601): Client=jenkins//172.31.14.131 stop 2023-02-13 05:01:18,898 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38439] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,38439,1676264474950' ***** 2023-02-13 05:01:18,898 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38439] regionserver.HRegionServer(2310): STOPPED: Stopped by RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38439 2023-02-13 05:01:18,898 DEBUG [M:0;jenkins-hbase4:38439] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5b5dd87, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 05:01:18,898 INFO [M:0;jenkins-hbase4:38439] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,38439,1676264474950 2023-02-13 05:01:18,901 DEBUG [M:0;jenkins-hbase4:38439] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6b26e56d to 127.0.0.1:60873 2023-02-13 05:01:18,901 DEBUG [M:0;jenkins-hbase4:38439] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:18,902 INFO [M:0;jenkins-hbase4:38439] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,38439,1676264474950; all regions closed. 2023-02-13 05:01:18,902 DEBUG [M:0;jenkins-hbase4:38439] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:18,902 DEBUG [M:0;jenkins-hbase4:38439] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 05:01:18,902 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 05:01:18,902 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676264477247] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676264477247,5,FailOnTimeoutGroup] 2023-02-13 05:01:18,902 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676264477247] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676264477247,5,FailOnTimeoutGroup] 2023-02-13 05:01:18,902 DEBUG [M:0;jenkins-hbase4:38439] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 05:01:18,903 INFO [M:0;jenkins-hbase4:38439] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 05:01:18,903 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 05:01:18,903 INFO [M:0;jenkins-hbase4:38439] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 05:01:18,904 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 05:01:18,904 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:18,904 INFO [M:0;jenkins-hbase4:38439] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-13 05:01:18,904 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:42931-0x1004c0defc80002, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 05:01:18,904 DEBUG [M:0;jenkins-hbase4:38439] master.HMaster(1502): Stopping service threads 2023-02-13 05:01:18,905 INFO [M:0;jenkins-hbase4:38439] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 05:01:18,905 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 05:01:18,905 INFO [M:0;jenkins-hbase4:38439] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 05:01:18,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42931-0x1004c0defc80002, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 05:01:18,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 05:01:18,908 ERROR [M:0;jenkins-hbase4:38439] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] 2023-02-13 05:01:18,908 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 05:01:18,908 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 05:01:18,908 INFO [Listener at localhost/37103] client.AsyncConnectionImpl(207): Connection has been closed by Listener at localhost/37103. 2023-02-13 05:01:18,909 DEBUG [Listener at localhost/37103] client.AsyncConnectionImpl(232): Call stack: at java.lang.Thread.getStackTrace(Thread.java:1564) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:209) at org.apache.hbase.thirdparty.com.google.common.io.Closeables.close(Closeables.java:79) at org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2.tearDown(TestAsyncClusterAdminApi2.java:75) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2023-02-13 05:01:18,909 INFO [M:0;jenkins-hbase4:38439] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 05:01:18,909 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 05:01:18,910 DEBUG [M:0;jenkins-hbase4:38439] zookeeper.ZKUtil(398): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 05:01:18,910 WARN [M:0;jenkins-hbase4:38439] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 05:01:18,910 INFO [M:0;jenkins-hbase4:38439] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 05:01:18,911 INFO [M:0;jenkins-hbase4:38439] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 05:01:18,912 DEBUG [M:0;jenkins-hbase4:38439] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 05:01:18,912 INFO [M:0;jenkins-hbase4:38439] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:18,912 DEBUG [M:0;jenkins-hbase4:38439] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:18,912 DEBUG [M:0;jenkins-hbase4:38439] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 05:01:18,912 DEBUG [M:0;jenkins-hbase4:38439] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:18,912 INFO [M:0;jenkins-hbase4:38439] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.07 KB heapSize=29.55 KB 2023-02-13 05:01:18,948 DEBUG [Listener at localhost/37103] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:18,949 DEBUG [Listener at localhost/37103] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2700073d to 127.0.0.1:60873 2023-02-13 05:01:18,951 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-13 05:01:18,952 DEBUG [Listener at localhost/37103] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6d08d536 to 127.0.0.1:60873 2023-02-13 05:01:18,952 DEBUG [Listener at localhost/37103] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:18,957 DEBUG [Listener at localhost/37103] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-13 05:01:18,957 INFO [Listener at localhost/37103] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,42931,1676264476125' ***** 2023-02-13 05:01:18,957 INFO [Listener at localhost/37103] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 05:01:18,957 INFO [Listener at localhost/37103] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33715,1676264476159' ***** 2023-02-13 05:01:18,957 INFO [Listener at localhost/37103] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 05:01:18,957 INFO [RS:1;jenkins-hbase4:42931] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 05:01:18,957 INFO [RS:2;jenkins-hbase4:33715] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 05:01:18,957 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 05:01:18,958 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 05:01:18,958 INFO [RS:2;jenkins-hbase4:33715] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 05:01:18,957 INFO [RS:1;jenkins-hbase4:42931] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 05:01:18,959 INFO [RS:2;jenkins-hbase4:33715] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 05:01:18,959 INFO [RS:1;jenkins-hbase4:42931] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 05:01:18,959 INFO [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33715,1676264476159 2023-02-13 05:01:18,959 INFO [RS:1;jenkins-hbase4:42931] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,42931,1676264476125 2023-02-13 05:01:18,959 DEBUG [RS:2;jenkins-hbase4:33715] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x71b88deb to 127.0.0.1:60873 2023-02-13 05:01:18,959 DEBUG [RS:1;jenkins-hbase4:42931] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x03dd11f3 to 127.0.0.1:60873 2023-02-13 05:01:18,959 DEBUG [RS:2;jenkins-hbase4:33715] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:18,959 DEBUG [RS:1;jenkins-hbase4:42931] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:18,959 INFO [RS:2;jenkins-hbase4:33715] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 05:01:18,959 INFO [RS:2;jenkins-hbase4:33715] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 05:01:18,959 INFO [RS:1;jenkins-hbase4:42931] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,42931,1676264476125; all regions closed. 2023-02-13 05:01:18,959 INFO [RS:2;jenkins-hbase4:33715] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 05:01:18,960 INFO [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 05:01:18,960 INFO [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 05:01:18,960 DEBUG [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-13 05:01:18,960 DEBUG [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 05:01:18,961 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 05:01:18,961 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 05:01:18,961 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 05:01:18,961 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 05:01:18,961 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 05:01:18,961 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-02-13 05:01:18,980 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 05:01:18,980 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 05:01:18,985 DEBUG [RS:1;jenkins-hbase4:42931] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/oldWALs 2023-02-13 05:01:18,985 INFO [RS:1;jenkins-hbase4:42931] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42931%2C1676264476125:(num 1676264477645) 2023-02-13 05:01:18,985 DEBUG [RS:1;jenkins-hbase4:42931] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:18,985 INFO [RS:1;jenkins-hbase4:42931] regionserver.LeaseManager(133): Closed leases 2023-02-13 05:01:18,985 INFO [RS:1;jenkins-hbase4:42931] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 05:01:18,986 INFO [RS:1;jenkins-hbase4:42931] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 05:01:18,986 INFO [RS:1;jenkins-hbase4:42931] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 05:01:18,986 INFO [RS:1;jenkins-hbase4:42931] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 05:01:18,986 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 05:01:18,987 INFO [RS:1;jenkins-hbase4:42931] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42931 2023-02-13 05:01:18,993 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42931,1676264476125 2023-02-13 05:01:18,993 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42931,1676264476125 2023-02-13 05:01:18,993 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 05:01:18,993 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 05:01:18,993 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 05:01:18,994 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:42931-0x1004c0defc80002, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42931,1676264476125 2023-02-13 05:01:18,994 ERROR [Listener at localhost/37103-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@124d3be5 rejected from java.util.concurrent.ThreadPoolExecutor@1672ad74[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:18,994 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:42931-0x1004c0defc80002, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 05:01:18,994 ERROR [Listener at localhost/37103-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@19807752 rejected from java.util.concurrent.ThreadPoolExecutor@1672ad74[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:19,041 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/namespace/00f7d0808277ee6962f65d76422c27a7/.tmp/info/b2ab4f4e5509492793789bc9e92d4610 2023-02-13 05:01:19,042 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/.tmp/info/eccd461a833f44a799f1b957ccf688e5 2023-02-13 05:01:19,042 INFO [M:0;jenkins-hbase4:38439] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.07 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/db53479daf2b4cdb804f923a140cc7fc 2023-02-13 05:01:19,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/namespace/00f7d0808277ee6962f65d76422c27a7/.tmp/info/b2ab4f4e5509492793789bc9e92d4610 as hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/namespace/00f7d0808277ee6962f65d76422c27a7/info/b2ab4f4e5509492793789bc9e92d4610 2023-02-13 05:01:19,084 DEBUG [M:0;jenkins-hbase4:38439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/db53479daf2b4cdb804f923a140cc7fc as hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/db53479daf2b4cdb804f923a140cc7fc 2023-02-13 05:01:19,085 DEBUG [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(1505): Waiting on 00f7d0808277ee6962f65d76422c27a7 2023-02-13 05:01:19,099 INFO [M:0;jenkins-hbase4:38439] regionserver.HStore(1080): Added hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/db53479daf2b4cdb804f923a140cc7fc, entries=8, sequenceid=66, filesize=6.3 K 2023-02-13 05:01:19,100 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/namespace/00f7d0808277ee6962f65d76422c27a7/info/b2ab4f4e5509492793789bc9e92d4610, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 05:01:19,101 INFO [M:0;jenkins-hbase4:38439] regionserver.HRegion(2947): Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 189ms, sequenceid=66, compaction requested=false 2023-02-13 05:01:19,111 INFO [M:0;jenkins-hbase4:38439] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:19,111 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 00f7d0808277ee6962f65d76422c27a7 in 228ms, sequenceid=6, compaction requested=false 2023-02-13 05:01:19,111 DEBUG [M:0;jenkins-hbase4:38439] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676264478912Disabling compacts and flushes for region at 1676264478912Disabling writes for close at 1676264478912Obtaining lock to block concurrent updates at 1676264478912Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676264478913 (+1 ms)Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=24646, getHeapSize=30248, getOffHeapSize=0, getCellsCount=71 at 1676264478913Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676264478914 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676264478915 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676264478985 (+70 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676264478989 (+4 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676264479087 (+98 ms)Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 189ms, sequenceid=66, compaction requested=false at 1676264479101 (+14 ms)Writing region close event to WAL at 1676264479111 (+10 ms)Closed at 1676264479111 2023-02-13 05:01:19,114 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-13 05:01:19,117 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/.tmp/table/a4309f0ed4084f629af3726e65812159 2023-02-13 05:01:19,124 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 05:01:19,124 INFO [M:0;jenkins-hbase4:38439] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 05:01:19,125 INFO [M:0;jenkins-hbase4:38439] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38439 2023-02-13 05:01:19,130 DEBUG [M:0;jenkins-hbase4:38439] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,38439,1676264474950 already deleted, retry=false 2023-02-13 05:01:19,130 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/namespace/00f7d0808277ee6962f65d76422c27a7/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-13 05:01:19,132 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7. 2023-02-13 05:01:19,132 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 00f7d0808277ee6962f65d76422c27a7: Waiting for close lock at 1676264478882Running coprocessor pre-close hooks at 1676264478882Disabling compacts and flushes for region at 1676264478882Disabling writes for close at 1676264478883 (+1 ms)Obtaining lock to block concurrent updates at 1676264478884 (+1 ms)Preparing flush snapshotting stores in 00f7d0808277ee6962f65d76422c27a7 at 1676264478884Finished memstore snapshotting hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676264478898 (+14 ms)Flushing stores of hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7. at 1676264478902 (+4 ms)Flushing 00f7d0808277ee6962f65d76422c27a7/info: creating writer at 1676264478909 (+7 ms)Flushing 00f7d0808277ee6962f65d76422c27a7/info: appending metadata at 1676264478983 (+74 ms)Flushing 00f7d0808277ee6962f65d76422c27a7/info: closing flushed file at 1676264478989 (+6 ms)Flushing 00f7d0808277ee6962f65d76422c27a7/info: reopening flushed file at 1676264479087 (+98 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 00f7d0808277ee6962f65d76422c27a7 in 228ms, sequenceid=6, compaction requested=false at 1676264479111 (+24 ms)Writing region close event to WAL at 1676264479122 (+11 ms)Running coprocessor post-close hooks at 1676264479131 (+9 ms)Closed at 1676264479131 2023-02-13 05:01:19,132 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676264478011.00f7d0808277ee6962f65d76422c27a7. 2023-02-13 05:01:19,135 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/.tmp/info/eccd461a833f44a799f1b957ccf688e5 as hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/info/eccd461a833f44a799f1b957ccf688e5 2023-02-13 05:01:19,143 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/info/eccd461a833f44a799f1b957ccf688e5, entries=10, sequenceid=9, filesize=5.9 K 2023-02-13 05:01:19,145 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/.tmp/table/a4309f0ed4084f629af3726e65812159 as hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/table/a4309f0ed4084f629af3726e65812159 2023-02-13 05:01:19,155 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/table/a4309f0ed4084f629af3726e65812159, entries=2, sequenceid=9, filesize=4.7 K 2023-02-13 05:01:19,157 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 195ms, sequenceid=9, compaction requested=false 2023-02-13 05:01:19,157 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-13 05:01:19,161 DEBUG [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 05:01:19,166 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-02-13 05:01:19,167 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 05:01:19,168 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 05:01:19,168 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676264478961Running coprocessor pre-close hooks at 1676264478961Disabling compacts and flushes for region at 1676264478961Disabling writes for close at 1676264478961Obtaining lock to block concurrent updates at 1676264478961Preparing flush snapshotting stores in 1588230740 at 1676264478961Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=1290, getHeapSize=2912, getOffHeapSize=0, getCellsCount=12 at 1676264478962 (+1 ms)Flushing stores of hbase:meta,,1.1588230740 at 1676264478964 (+2 ms)Flushing 1588230740/info: creating writer at 1676264478964Flushing 1588230740/info: appending metadata at 1676264478982 (+18 ms)Flushing 1588230740/info: closing flushed file at 1676264478989 (+7 ms)Flushing 1588230740/table: creating writer at 1676264479083 (+94 ms)Flushing 1588230740/table: appending metadata at 1676264479089 (+6 ms)Flushing 1588230740/table: closing flushed file at 1676264479089Flushing 1588230740/info: reopening flushed file at 1676264479136 (+47 ms)Flushing 1588230740/table: reopening flushed file at 1676264479145 (+9 ms)Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 195ms, sequenceid=9, compaction requested=false at 1676264479157 (+12 ms)Writing region close event to WAL at 1676264479161 (+4 ms)Running coprocessor post-close hooks at 1676264479167 (+6 ms)Closed at 1676264479168 (+1 ms) 2023-02-13 05:01:19,168 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 05:01:19,285 INFO [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,37253,1676264476079; all regions closed. 2023-02-13 05:01:19,294 DEBUG [RS:0;jenkins-hbase4:37253] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/oldWALs 2023-02-13 05:01:19,294 INFO [RS:0;jenkins-hbase4:37253] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37253%2C1676264476079:(num 1676264477645) 2023-02-13 05:01:19,294 DEBUG [RS:0;jenkins-hbase4:37253] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:19,294 INFO [RS:0;jenkins-hbase4:37253] regionserver.LeaseManager(133): Closed leases 2023-02-13 05:01:19,294 INFO [RS:0;jenkins-hbase4:37253] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 05:01:19,294 INFO [RS:0;jenkins-hbase4:37253] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 05:01:19,294 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 05:01:19,294 INFO [RS:0;jenkins-hbase4:37253] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 05:01:19,294 INFO [RS:0;jenkins-hbase4:37253] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 05:01:19,295 INFO [RS:0;jenkins-hbase4:37253] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37253 2023-02-13 05:01:19,299 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37253,1676264476079 2023-02-13 05:01:19,299 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37253,1676264476079 2023-02-13 05:01:19,299 ERROR [Listener at localhost/37103-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7d8f3b3b rejected from java.util.concurrent.ThreadPoolExecutor@79b0a052[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:19,361 INFO [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33715,1676264476159; all regions closed. 2023-02-13 05:01:19,369 DEBUG [RS:2;jenkins-hbase4:33715] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/oldWALs 2023-02-13 05:01:19,369 INFO [RS:2;jenkins-hbase4:33715] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33715%2C1676264476159.meta:.meta(num 1676264477744) 2023-02-13 05:01:19,376 DEBUG [RS:2;jenkins-hbase4:33715] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/oldWALs 2023-02-13 05:01:19,376 INFO [RS:2;jenkins-hbase4:33715] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33715%2C1676264476159:(num 1676264477645) 2023-02-13 05:01:19,376 DEBUG [RS:2;jenkins-hbase4:33715] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:19,376 INFO [RS:2;jenkins-hbase4:33715] regionserver.LeaseManager(133): Closed leases 2023-02-13 05:01:19,376 INFO [RS:2;jenkins-hbase4:33715] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 05:01:19,377 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 05:01:19,377 INFO [RS:2;jenkins-hbase4:33715] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33715 2023-02-13 05:01:19,379 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33715,1676264476159 2023-02-13 05:01:19,379 ERROR [Listener at localhost/37103-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@11ca02eb rejected from java.util.concurrent.ThreadPoolExecutor@15f5940a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:19,586 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:19,586 INFO [RS:2;jenkins-hbase4:33715] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33715,1676264476159; zookeeper connection closed. 2023-02-13 05:01:19,586 ERROR [Listener at localhost/37103-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2071cb9b rejected from java.util.concurrent.ThreadPoolExecutor@15f5940a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:19,587 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:33715-0x1004c0defc80003, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:19,587 ERROR [Listener at localhost/37103-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5801c5c7 rejected from java.util.concurrent.ThreadPoolExecutor@15f5940a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:19,587 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4014029e] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4014029e 2023-02-13 05:01:19,686 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:19,686 INFO [RS:0;jenkins-hbase4:37253] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,37253,1676264476079; zookeeper connection closed. 2023-02-13 05:01:19,687 ERROR [Listener at localhost/37103-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@996ec6f rejected from java.util.concurrent.ThreadPoolExecutor@79b0a052[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:19,687 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7c9e39b5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7c9e39b5 2023-02-13 05:01:19,687 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:37253-0x1004c0defc80001, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:19,687 ERROR [Listener at localhost/37103-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@718acb4f rejected from java.util.concurrent.ThreadPoolExecutor@79b0a052[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:19,787 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:19,787 INFO [M:0;jenkins-hbase4:38439] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,38439,1676264474950; zookeeper connection closed. 2023-02-13 05:01:19,787 ERROR [Listener at localhost/37103-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6db08e3f rejected from java.util.concurrent.ThreadPoolExecutor@505bdbd7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 24] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:19,787 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): master:38439-0x1004c0defc80000, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:19,787 ERROR [Listener at localhost/37103-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2cfc9e1f rejected from java.util.concurrent.ThreadPoolExecutor@505bdbd7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 24] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:19,887 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:42931-0x1004c0defc80002, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:19,887 INFO [RS:1;jenkins-hbase4:42931] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,42931,1676264476125; zookeeper connection closed. 2023-02-13 05:01:19,887 ERROR [Listener at localhost/37103-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@61f3fdb0 rejected from java.util.concurrent.ThreadPoolExecutor@1672ad74[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:19,887 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@cd42595] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@cd42595 2023-02-13 05:01:19,887 DEBUG [Listener at localhost/37103-EventThread] zookeeper.ZKWatcher(600): regionserver:42931-0x1004c0defc80002, quorum=127.0.0.1:60873, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:19,887 ERROR [Listener at localhost/37103-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@29ff154f rejected from java.util.concurrent.ThreadPoolExecutor@1672ad74[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:19,888 INFO [Listener at localhost/37103] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-13 05:01:19,888 WARN [Listener at localhost/37103] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 05:01:19,891 INFO [Listener at localhost/37103] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 05:01:19,993 WARN [BP-95129621-172.31.14.131-1676264471844 heartbeating to localhost/127.0.0.1:43091] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 05:01:19,993 WARN [BP-95129621-172.31.14.131-1676264471844 heartbeating to localhost/127.0.0.1:43091] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-95129621-172.31.14.131-1676264471844 (Datanode Uuid 4d409657-b597-4a45-bf1d-41673fe1d75c) service to localhost/127.0.0.1:43091 2023-02-13 05:01:19,995 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/cluster_f3f6ca08-de39-71f7-ec28-b21a04d3bd2a/dfs/data/data5/current/BP-95129621-172.31.14.131-1676264471844] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 05:01:19,995 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/cluster_f3f6ca08-de39-71f7-ec28-b21a04d3bd2a/dfs/data/data6/current/BP-95129621-172.31.14.131-1676264471844] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 05:01:19,996 WARN [Listener at localhost/37103] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 05:01:19,998 INFO [Listener at localhost/37103] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 05:01:20,100 WARN [BP-95129621-172.31.14.131-1676264471844 heartbeating to localhost/127.0.0.1:43091] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 05:01:20,100 WARN [BP-95129621-172.31.14.131-1676264471844 heartbeating to localhost/127.0.0.1:43091] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-95129621-172.31.14.131-1676264471844 (Datanode Uuid 255ddfb9-4f75-4ad5-b387-eee2121ba847) service to localhost/127.0.0.1:43091 2023-02-13 05:01:20,101 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/cluster_f3f6ca08-de39-71f7-ec28-b21a04d3bd2a/dfs/data/data3/current/BP-95129621-172.31.14.131-1676264471844] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 05:01:20,101 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/cluster_f3f6ca08-de39-71f7-ec28-b21a04d3bd2a/dfs/data/data4/current/BP-95129621-172.31.14.131-1676264471844] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 05:01:20,101 WARN [Listener at localhost/37103] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 05:01:20,103 INFO [Listener at localhost/37103] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 05:01:20,206 WARN [BP-95129621-172.31.14.131-1676264471844 heartbeating to localhost/127.0.0.1:43091] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 05:01:20,206 WARN [BP-95129621-172.31.14.131-1676264471844 heartbeating to localhost/127.0.0.1:43091] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-95129621-172.31.14.131-1676264471844 (Datanode Uuid 44841879-a436-44ba-8bbb-055c9c1a883f) service to localhost/127.0.0.1:43091 2023-02-13 05:01:20,206 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/cluster_f3f6ca08-de39-71f7-ec28-b21a04d3bd2a/dfs/data/data1/current/BP-95129621-172.31.14.131-1676264471844] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 05:01:20,207 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/cluster_f3f6ca08-de39-71f7-ec28-b21a04d3bd2a/dfs/data/data2/current/BP-95129621-172.31.14.131-1676264471844] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 05:01:20,236 INFO [Listener at localhost/37103] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 05:01:20,355 INFO [Listener at localhost/37103] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-13 05:01:20,395 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-02-13 05:01:20,408 INFO [Listener at localhost/37103] hbase.ResourceChecker(175): after: client.TestAsyncClusterAdminApi2#testStop Thread=76 (was 8) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1171269600) connection to localhost/127.0.0.1:43091 from jenkins.hfs.1 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@e31a3db java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:43091 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1171269600) connection to localhost/127.0.0.1:43091 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: IPC Client (1171269600) connection to localhost/127.0.0.1:43091 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: ForkJoinPool-2-worker-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-5-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1171269600) connection to localhost/127.0.0.1:43091 from jenkins.hfs.0 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:300) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost:43091 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/37103 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1171269600) connection to localhost/127.0.0.1:43091 from jenkins.hfs.2 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost:43091 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:43091 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-5-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-5-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'NameNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=506 (was 260) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=190 (was 150) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3625 (was 4203) 2023-02-13 05:01:20,417 INFO [Listener at localhost/37103] hbase.ResourceChecker(147): before: client.TestAsyncClusterAdminApi2#testShutdown Thread=76, OpenFileDescriptor=506, MaxFileDescriptor=60000, SystemLoadAverage=190, ProcessCount=168, AvailableMemoryMB=3625 2023-02-13 05:01:20,417 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-13 05:01:20,418 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/hadoop.log.dir so I do NOT create it in target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f 2023-02-13 05:01:20,418 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af8f8d74-b2b9-7b24-088b-0696e9e138d2/hadoop.tmp.dir so I do NOT create it in target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f 2023-02-13 05:01:20,418 INFO [Listener at localhost/37103] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/cluster_bcbf8140-8def-93b2-7aef-cc1ac3cf1091, deleteOnExit=true 2023-02-13 05:01:20,418 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-13 05:01:20,418 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/test.cache.data in system properties and HBase conf 2023-02-13 05:01:20,419 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/hadoop.tmp.dir in system properties and HBase conf 2023-02-13 05:01:20,419 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/hadoop.log.dir in system properties and HBase conf 2023-02-13 05:01:20,419 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-13 05:01:20,419 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-13 05:01:20,419 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-13 05:01:20,420 DEBUG [Listener at localhost/37103] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-13 05:01:20,420 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-13 05:01:20,420 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-13 05:01:20,421 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-13 05:01:20,421 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 05:01:20,421 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-13 05:01:20,421 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-13 05:01:20,421 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 05:01:20,422 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 05:01:20,422 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-13 05:01:20,422 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/nfs.dump.dir in system properties and HBase conf 2023-02-13 05:01:20,422 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/java.io.tmpdir in system properties and HBase conf 2023-02-13 05:01:20,422 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 05:01:20,423 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-13 05:01:20,423 INFO [Listener at localhost/37103] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-13 05:01:20,428 WARN [Listener at localhost/37103] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 05:01:20,429 WARN [Listener at localhost/37103] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 05:01:20,471 WARN [Listener at localhost/37103] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 05:01:20,475 INFO [Listener at localhost/37103] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 05:01:20,479 INFO [Listener at localhost/37103] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/java.io.tmpdir/Jetty_localhost_38775_hdfs____.tmnddy/webapp 2023-02-13 05:01:20,571 INFO [Listener at localhost/37103] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38775 2023-02-13 05:01:20,575 WARN [Listener at localhost/37103] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 05:01:20,575 WARN [Listener at localhost/37103] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 05:01:20,621 WARN [Listener at localhost/38635] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 05:01:20,632 WARN [Listener at localhost/38635] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 05:01:20,634 WARN [Listener at localhost/38635] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 05:01:20,635 INFO [Listener at localhost/38635] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 05:01:20,640 INFO [Listener at localhost/38635] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/java.io.tmpdir/Jetty_localhost_42609_datanode____v56go8/webapp 2023-02-13 05:01:20,730 INFO [Listener at localhost/38635] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42609 2023-02-13 05:01:20,738 WARN [Listener at localhost/38919] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 05:01:20,757 WARN [Listener at localhost/38919] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 05:01:20,759 WARN [Listener at localhost/38919] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 05:01:20,761 INFO [Listener at localhost/38919] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 05:01:20,764 INFO [Listener at localhost/38919] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/java.io.tmpdir/Jetty_localhost_44561_datanode____.m0sqnd/webapp 2023-02-13 05:01:20,841 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x8e899358202bb35b: Processing first storage report for DS-d844d497-cdc4-4905-a59b-c5a65740ae3c from datanode 98cfa102-ef1a-4f17-b112-c1df27aaca41 2023-02-13 05:01:20,842 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x8e899358202bb35b: from storage DS-d844d497-cdc4-4905-a59b-c5a65740ae3c node DatanodeRegistration(127.0.0.1:43865, datanodeUuid=98cfa102-ef1a-4f17-b112-c1df27aaca41, infoPort=40507, infoSecurePort=0, ipcPort=38919, storageInfo=lv=-57;cid=testClusterID;nsid=806133223;c=1676264480432), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 05:01:20,842 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x8e899358202bb35b: Processing first storage report for DS-727c22c8-368e-41ca-8cae-ba02c70f97a9 from datanode 98cfa102-ef1a-4f17-b112-c1df27aaca41 2023-02-13 05:01:20,842 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x8e899358202bb35b: from storage DS-727c22c8-368e-41ca-8cae-ba02c70f97a9 node DatanodeRegistration(127.0.0.1:43865, datanodeUuid=98cfa102-ef1a-4f17-b112-c1df27aaca41, infoPort=40507, infoSecurePort=0, ipcPort=38919, storageInfo=lv=-57;cid=testClusterID;nsid=806133223;c=1676264480432), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 05:01:20,877 INFO [Listener at localhost/38919] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44561 2023-02-13 05:01:20,887 WARN [Listener at localhost/39851] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 05:01:20,902 WARN [Listener at localhost/39851] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 05:01:20,904 WARN [Listener at localhost/39851] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 05:01:20,905 INFO [Listener at localhost/39851] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 05:01:20,910 INFO [Listener at localhost/39851] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/java.io.tmpdir/Jetty_localhost_39969_datanode____.ppkcjn/webapp 2023-02-13 05:01:20,994 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6e95d23403233be1: Processing first storage report for DS-19598a71-fd50-4434-9250-49a2527e4f19 from datanode 762c31f2-9324-439e-94b6-570be9285088 2023-02-13 05:01:20,995 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6e95d23403233be1: from storage DS-19598a71-fd50-4434-9250-49a2527e4f19 node DatanodeRegistration(127.0.0.1:43243, datanodeUuid=762c31f2-9324-439e-94b6-570be9285088, infoPort=44279, infoSecurePort=0, ipcPort=39851, storageInfo=lv=-57;cid=testClusterID;nsid=806133223;c=1676264480432), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-13 05:01:20,995 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6e95d23403233be1: Processing first storage report for DS-026dab6a-c42d-4ff6-b971-67e5c0462117 from datanode 762c31f2-9324-439e-94b6-570be9285088 2023-02-13 05:01:20,995 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6e95d23403233be1: from storage DS-026dab6a-c42d-4ff6-b971-67e5c0462117 node DatanodeRegistration(127.0.0.1:43243, datanodeUuid=762c31f2-9324-439e-94b6-570be9285088, infoPort=44279, infoSecurePort=0, ipcPort=39851, storageInfo=lv=-57;cid=testClusterID;nsid=806133223;c=1676264480432), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 05:01:21,018 INFO [Listener at localhost/39851] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39969 2023-02-13 05:01:21,027 WARN [Listener at localhost/35553] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 05:01:21,130 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6186022ca4593505: Processing first storage report for DS-8cd3cfbb-f10f-4864-be3a-1bc6b43884e9 from datanode 2d27b5c6-3da9-420d-9bfa-721f4684c8c2 2023-02-13 05:01:21,130 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6186022ca4593505: from storage DS-8cd3cfbb-f10f-4864-be3a-1bc6b43884e9 node DatanodeRegistration(127.0.0.1:36757, datanodeUuid=2d27b5c6-3da9-420d-9bfa-721f4684c8c2, infoPort=33893, infoSecurePort=0, ipcPort=35553, storageInfo=lv=-57;cid=testClusterID;nsid=806133223;c=1676264480432), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 05:01:21,130 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6186022ca4593505: Processing first storage report for DS-8496ea5c-de48-4902-a9cc-14dad48f44d8 from datanode 2d27b5c6-3da9-420d-9bfa-721f4684c8c2 2023-02-13 05:01:21,130 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6186022ca4593505: from storage DS-8496ea5c-de48-4902-a9cc-14dad48f44d8 node DatanodeRegistration(127.0.0.1:36757, datanodeUuid=2d27b5c6-3da9-420d-9bfa-721f4684c8c2, infoPort=33893, infoSecurePort=0, ipcPort=35553, storageInfo=lv=-57;cid=testClusterID;nsid=806133223;c=1676264480432), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 05:01:21,138 DEBUG [Listener at localhost/35553] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f 2023-02-13 05:01:21,140 INFO [Listener at localhost/35553] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/cluster_bcbf8140-8def-93b2-7aef-cc1ac3cf1091/zookeeper_0, clientPort=61432, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/cluster_bcbf8140-8def-93b2-7aef-cc1ac3cf1091/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/cluster_bcbf8140-8def-93b2-7aef-cc1ac3cf1091/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-13 05:01:21,144 INFO [Listener at localhost/35553] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=61432 2023-02-13 05:01:21,145 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:21,146 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:21,171 INFO [Listener at localhost/35553] util.FSUtils(479): Created version file at hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a with version=8 2023-02-13 05:01:21,172 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:43091/user/jenkins/test-data/a9a50529-f377-590a-ce8b-98a9cb353c22/hbase-staging 2023-02-13 05:01:21,173 INFO [Listener at localhost/35553] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=6 2023-02-13 05:01:21,174 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:21,174 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:21,174 INFO [Listener at localhost/35553] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 05:01:21,174 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:21,174 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 05:01:21,174 INFO [Listener at localhost/35553] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 05:01:21,177 INFO [Listener at localhost/35553] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37449 2023-02-13 05:01:21,177 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:21,179 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:21,180 INFO [Listener at localhost/35553] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37449 connecting to ZooKeeper ensemble=127.0.0.1:61432 2023-02-13 05:01:21,188 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:374490x0, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 05:01:21,189 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:37449-0x1004c0e0b390000 connected 2023-02-13 05:01:21,208 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 05:01:21,209 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 05:01:21,209 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 05:01:21,210 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37449 2023-02-13 05:01:21,210 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37449 2023-02-13 05:01:21,210 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37449 2023-02-13 05:01:21,211 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37449 2023-02-13 05:01:21,211 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37449 2023-02-13 05:01:21,211 INFO [Listener at localhost/35553] master.HMaster(439): hbase.rootdir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a, hbase.cluster.distributed=false 2023-02-13 05:01:21,224 INFO [Listener at localhost/35553] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-13 05:01:21,225 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:21,225 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:21,225 INFO [Listener at localhost/35553] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 05:01:21,225 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:21,225 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 05:01:21,225 INFO [Listener at localhost/35553] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 05:01:21,226 INFO [Listener at localhost/35553] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39207 2023-02-13 05:01:21,227 INFO [Listener at localhost/35553] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 05:01:21,228 DEBUG [Listener at localhost/35553] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 05:01:21,229 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:21,230 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:21,231 INFO [Listener at localhost/35553] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39207 connecting to ZooKeeper ensemble=127.0.0.1:61432 2023-02-13 05:01:21,234 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:392070x0, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 05:01:21,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:39207-0x1004c0e0b390001 connected 2023-02-13 05:01:21,235 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): regionserver:39207-0x1004c0e0b390001, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 05:01:21,236 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): regionserver:39207-0x1004c0e0b390001, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 05:01:21,236 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): regionserver:39207-0x1004c0e0b390001, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 05:01:21,237 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39207 2023-02-13 05:01:21,238 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39207 2023-02-13 05:01:21,238 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39207 2023-02-13 05:01:21,241 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39207 2023-02-13 05:01:21,243 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39207 2023-02-13 05:01:21,254 INFO [Listener at localhost/35553] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-13 05:01:21,254 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:21,254 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:21,255 INFO [Listener at localhost/35553] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 05:01:21,255 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:21,255 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 05:01:21,255 INFO [Listener at localhost/35553] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 05:01:21,256 INFO [Listener at localhost/35553] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45019 2023-02-13 05:01:21,257 INFO [Listener at localhost/35553] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 05:01:21,258 DEBUG [Listener at localhost/35553] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 05:01:21,258 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:21,259 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:21,260 INFO [Listener at localhost/35553] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45019 connecting to ZooKeeper ensemble=127.0.0.1:61432 2023-02-13 05:01:21,263 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:450190x0, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 05:01:21,264 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): regionserver:450190x0, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 05:01:21,265 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:45019-0x1004c0e0b390002 connected 2023-02-13 05:01:21,265 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 05:01:21,266 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 05:01:21,268 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45019 2023-02-13 05:01:21,268 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45019 2023-02-13 05:01:21,268 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45019 2023-02-13 05:01:21,272 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45019 2023-02-13 05:01:21,272 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45019 2023-02-13 05:01:21,283 INFO [Listener at localhost/35553] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=6 2023-02-13 05:01:21,283 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:21,283 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:21,283 INFO [Listener at localhost/35553] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 05:01:21,283 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 05:01:21,284 INFO [Listener at localhost/35553] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 05:01:21,284 INFO [Listener at localhost/35553] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 05:01:21,285 INFO [Listener at localhost/35553] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42641 2023-02-13 05:01:21,285 INFO [Listener at localhost/35553] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 05:01:21,287 DEBUG [Listener at localhost/35553] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 05:01:21,288 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:21,289 INFO [Listener at localhost/35553] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:21,290 INFO [Listener at localhost/35553] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42641 connecting to ZooKeeper ensemble=127.0.0.1:61432 2023-02-13 05:01:21,292 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:426410x0, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 05:01:21,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:42641-0x1004c0e0b390003 connected 2023-02-13 05:01:21,294 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 05:01:21,294 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 05:01:21,295 DEBUG [Listener at localhost/35553] zookeeper.ZKUtil(164): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 05:01:21,295 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42641 2023-02-13 05:01:21,295 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42641 2023-02-13 05:01:21,296 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42641 2023-02-13 05:01:21,296 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42641 2023-02-13 05:01:21,296 DEBUG [Listener at localhost/35553] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42641 2023-02-13 05:01:21,297 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,37449,1676264481173 2023-02-13 05:01:21,300 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 05:01:21,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,37449,1676264481173 2023-02-13 05:01:21,302 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 05:01:21,302 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 05:01:21,302 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:21,302 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:39207-0x1004c0e0b390001, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 05:01:21,302 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 05:01:21,304 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 05:01:21,305 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 05:01:21,305 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,37449,1676264481173 from backup master directory 2023-02-13 05:01:21,306 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,37449,1676264481173 2023-02-13 05:01:21,306 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 05:01:21,306 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 05:01:21,306 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,37449,1676264481173 2023-02-13 05:01:21,326 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/hbase.id with ID: 8a152435-7ac9-45da-a54f-fe5a50cce947 2023-02-13 05:01:21,342 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 05:01:21,344 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:21,360 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x139f1282 to 127.0.0.1:61432 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 05:01:21,365 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@174c1506, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 05:01:21,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 05:01:21,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 05:01:21,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 05:01:21,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/MasterData/data/master/store-tmp 2023-02-13 05:01:21,382 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 05:01:21,382 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 05:01:21,382 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:21,382 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:21,382 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 05:01:21,382 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:21,382 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:21,382 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676264481382Disabling compacts and flushes for region at 1676264481382Disabling writes for close at 1676264481382Writing region close event to WAL at 1676264481382Closed at 1676264481382 2023-02-13 05:01:21,383 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/MasterData/WALs/jenkins-hbase4.apache.org,37449,1676264481173 2023-02-13 05:01:21,387 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37449%2C1676264481173, suffix=, logDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/MasterData/WALs/jenkins-hbase4.apache.org,37449,1676264481173, archiveDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/MasterData/oldWALs, maxLogs=10 2023-02-13 05:01:21,405 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43243,DS-19598a71-fd50-4434-9250-49a2527e4f19,DISK] 2023-02-13 05:01:21,407 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43865,DS-d844d497-cdc4-4905-a59b-c5a65740ae3c,DISK] 2023-02-13 05:01:21,408 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36757,DS-8cd3cfbb-f10f-4864-be3a-1bc6b43884e9,DISK] 2023-02-13 05:01:21,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/MasterData/WALs/jenkins-hbase4.apache.org,37449,1676264481173/jenkins-hbase4.apache.org%2C37449%2C1676264481173.1676264481387 2023-02-13 05:01:21,412 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43243,DS-19598a71-fd50-4434-9250-49a2527e4f19,DISK], DatanodeInfoWithStorage[127.0.0.1:43865,DS-d844d497-cdc4-4905-a59b-c5a65740ae3c,DISK], DatanodeInfoWithStorage[127.0.0.1:36757,DS-8cd3cfbb-f10f-4864-be3a-1bc6b43884e9,DISK]] 2023-02-13 05:01:21,412 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 05:01:21,412 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 05:01:21,412 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 05:01:21,412 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 05:01:21,415 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 05:01:21,417 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 05:01:21,417 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 05:01:21,418 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:21,419 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 05:01:21,420 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 05:01:21,423 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 05:01:21,426 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 05:01:21,426 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69289434, jitterRate=0.032493025064468384}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 05:01:21,426 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676264481412Initializing all the Stores at 1676264481414 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264481414Cleaning up temporary data from old regions at 1676264481420 (+6 ms)Cleaning up detritus from prior splits at 1676264481421 (+1 ms)Region opened successfully at 1676264481426 (+5 ms) 2023-02-13 05:01:21,427 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 05:01:21,428 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-13 05:01:21,428 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-13 05:01:21,428 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 05:01:21,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 05:01:21,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-02-13 05:01:21,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 05:01:21,431 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-13 05:01:21,433 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-13 05:01:21,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 05:01:21,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 05:01:21,443 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-13 05:01:21,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 05:01:21,444 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 05:01:21,447 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:21,447 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 05:01:21,448 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 05:01:21,449 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 05:01:21,450 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 05:01:21,450 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:39207-0x1004c0e0b390001, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 05:01:21,450 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:21,450 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 05:01:21,450 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 05:01:21,451 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,37449,1676264481173, sessionid=0x1004c0e0b390000, setting cluster-up flag (Was=false) 2023-02-13 05:01:21,456 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:21,461 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 05:01:21,462 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37449,1676264481173 2023-02-13 05:01:21,466 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:21,472 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 05:01:21,473 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37449,1676264481173 2023-02-13 05:01:21,474 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/.hbase-snapshot/.tmp 2023-02-13 05:01:21,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-13 05:01:21,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 05:01:21,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 05:01:21,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 05:01:21,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 05:01:21,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-13 05:01:21,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 05:01:21,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264511480 2023-02-13 05:01:21,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 05:01:21,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 05:01:21,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 05:01:21,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 05:01:21,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 05:01:21,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 05:01:21,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 05:01:21,482 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 05:01:21,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 05:01:21,482 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-13 05:01:21,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 05:01:21,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 05:01:21,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 05:01:21,483 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676264481483,5,FailOnTimeoutGroup] 2023-02-13 05:01:21,483 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 05:01:21,488 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676264481483,5,FailOnTimeoutGroup] 2023-02-13 05:01:21,488 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,488 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 05:01:21,488 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,488 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,500 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(952): ClusterId : 8a152435-7ac9-45da-a54f-fe5a50cce947 2023-02-13 05:01:21,500 INFO [RS:0;jenkins-hbase4:39207] regionserver.HRegionServer(952): ClusterId : 8a152435-7ac9-45da-a54f-fe5a50cce947 2023-02-13 05:01:21,501 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 05:01:21,501 DEBUG [RS:0;jenkins-hbase4:39207] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 05:01:21,500 INFO [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(952): ClusterId : 8a152435-7ac9-45da-a54f-fe5a50cce947 2023-02-13 05:01:21,502 DEBUG [RS:2;jenkins-hbase4:42641] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 05:01:21,506 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 05:01:21,506 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 05:01:21,506 DEBUG [RS:2;jenkins-hbase4:42641] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 05:01:21,506 DEBUG [RS:0;jenkins-hbase4:39207] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 05:01:21,507 DEBUG [RS:0;jenkins-hbase4:39207] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 05:01:21,507 DEBUG [RS:2;jenkins-hbase4:42641] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 05:01:21,508 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 05:01:21,508 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 05:01:21,509 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a 2023-02-13 05:01:21,511 DEBUG [RS:0;jenkins-hbase4:39207] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 05:01:21,511 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 05:01:21,511 DEBUG [RS:2;jenkins-hbase4:42641] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 05:01:21,513 DEBUG [RS:1;jenkins-hbase4:45019] zookeeper.ReadOnlyZKClient(139): Connect 0x04643388 to 127.0.0.1:61432 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 05:01:21,513 DEBUG [RS:2;jenkins-hbase4:42641] zookeeper.ReadOnlyZKClient(139): Connect 0x34e8d17a to 127.0.0.1:61432 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 05:01:21,513 DEBUG [RS:0;jenkins-hbase4:39207] zookeeper.ReadOnlyZKClient(139): Connect 0x03b6dd4f to 127.0.0.1:61432 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 05:01:21,520 DEBUG [RS:0;jenkins-hbase4:39207] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@78959e1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 05:01:21,520 DEBUG [RS:2;jenkins-hbase4:42641] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@30d7433f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 05:01:21,521 DEBUG [RS:0;jenkins-hbase4:39207] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@428bd774, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 05:01:21,521 DEBUG [RS:2;jenkins-hbase4:42641] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@75d5db41, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 05:01:21,523 DEBUG [RS:1;jenkins-hbase4:45019] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@455939b3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 05:01:21,523 DEBUG [RS:1;jenkins-hbase4:45019] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@518bcb08, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 05:01:21,529 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 05:01:21,531 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 05:01:21,533 DEBUG [RS:2;jenkins-hbase4:42641] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:42641 2023-02-13 05:01:21,533 INFO [RS:2;jenkins-hbase4:42641] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 05:01:21,533 INFO [RS:2;jenkins-hbase4:42641] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 05:01:21,533 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/info 2023-02-13 05:01:21,533 DEBUG [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 05:01:21,534 DEBUG [RS:0;jenkins-hbase4:39207] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:39207 2023-02-13 05:01:21,534 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 05:01:21,534 INFO [RS:0;jenkins-hbase4:39207] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 05:01:21,534 INFO [RS:0;jenkins-hbase4:39207] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 05:01:21,534 DEBUG [RS:1;jenkins-hbase4:45019] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:45019 2023-02-13 05:01:21,534 DEBUG [RS:0;jenkins-hbase4:39207] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 05:01:21,535 INFO [RS:1;jenkins-hbase4:45019] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 05:01:21,535 INFO [RS:1;jenkins-hbase4:45019] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 05:01:21,534 INFO [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37449,1676264481173 with isa=jenkins-hbase4.apache.org/172.31.14.131:42641, startcode=1676264481283 2023-02-13 05:01:21,535 DEBUG [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 05:01:21,535 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:21,535 DEBUG [RS:2;jenkins-hbase4:42641] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 05:01:21,535 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 05:01:21,535 INFO [RS:0;jenkins-hbase4:39207] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37449,1676264481173 with isa=jenkins-hbase4.apache.org/172.31.14.131:39207, startcode=1676264481224 2023-02-13 05:01:21,535 DEBUG [RS:0;jenkins-hbase4:39207] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 05:01:21,536 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37449,1676264481173 with isa=jenkins-hbase4.apache.org/172.31.14.131:45019, startcode=1676264481254 2023-02-13 05:01:21,536 DEBUG [RS:1;jenkins-hbase4:45019] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 05:01:21,543 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45119, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 05:01:21,544 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51343, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 05:01:21,544 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40283, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 05:01:21,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37449] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42641,1676264481283 2023-02-13 05:01:21,548 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37449] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,39207,1676264481224 2023-02-13 05:01:21,548 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37449] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:21,549 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/rep_barrier 2023-02-13 05:01:21,549 DEBUG [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a 2023-02-13 05:01:21,549 DEBUG [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38635 2023-02-13 05:01:21,549 DEBUG [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a 2023-02-13 05:01:21,549 DEBUG [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 05:01:21,549 DEBUG [RS:0;jenkins-hbase4:39207] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a 2023-02-13 05:01:21,549 DEBUG [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38635 2023-02-13 05:01:21,549 DEBUG [RS:0;jenkins-hbase4:39207] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38635 2023-02-13 05:01:21,549 DEBUG [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 05:01:21,549 DEBUG [RS:0;jenkins-hbase4:39207] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 05:01:21,549 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 05:01:21,550 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:21,551 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 05:01:21,551 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 05:01:21,556 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/table 2023-02-13 05:01:21,556 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,39207,1676264481224] 2023-02-13 05:01:21,556 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45019,1676264481254] 2023-02-13 05:01:21,556 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42641,1676264481283] 2023-02-13 05:01:21,556 DEBUG [RS:1;jenkins-hbase4:45019] zookeeper.ZKUtil(162): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:21,556 DEBUG [RS:0;jenkins-hbase4:39207] zookeeper.ZKUtil(162): regionserver:39207-0x1004c0e0b390001, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39207,1676264481224 2023-02-13 05:01:21,556 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 05:01:21,556 WARN [RS:0;jenkins-hbase4:39207] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 05:01:21,556 WARN [RS:1;jenkins-hbase4:45019] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 05:01:21,557 INFO [RS:0;jenkins-hbase4:39207] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 05:01:21,557 DEBUG [RS:2;jenkins-hbase4:42641] zookeeper.ZKUtil(162): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42641,1676264481283 2023-02-13 05:01:21,557 DEBUG [RS:0;jenkins-hbase4:39207] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/WALs/jenkins-hbase4.apache.org,39207,1676264481224 2023-02-13 05:01:21,557 INFO [RS:1;jenkins-hbase4:45019] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 05:01:21,557 WARN [RS:2;jenkins-hbase4:42641] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 05:01:21,557 DEBUG [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/WALs/jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:21,558 INFO [RS:2;jenkins-hbase4:42641] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 05:01:21,558 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:21,558 DEBUG [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/WALs/jenkins-hbase4.apache.org,42641,1676264481283 2023-02-13 05:01:21,559 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740 2023-02-13 05:01:21,561 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740 2023-02-13 05:01:21,564 DEBUG [RS:0;jenkins-hbase4:39207] zookeeper.ZKUtil(162): regionserver:39207-0x1004c0e0b390001, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:21,564 DEBUG [RS:0;jenkins-hbase4:39207] zookeeper.ZKUtil(162): regionserver:39207-0x1004c0e0b390001, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39207,1676264481224 2023-02-13 05:01:21,564 DEBUG [RS:2;jenkins-hbase4:42641] zookeeper.ZKUtil(162): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:21,564 DEBUG [RS:1;jenkins-hbase4:45019] zookeeper.ZKUtil(162): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:21,565 DEBUG [RS:0;jenkins-hbase4:39207] zookeeper.ZKUtil(162): regionserver:39207-0x1004c0e0b390001, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42641,1676264481283 2023-02-13 05:01:21,565 DEBUG [RS:2;jenkins-hbase4:42641] zookeeper.ZKUtil(162): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39207,1676264481224 2023-02-13 05:01:21,565 DEBUG [RS:1;jenkins-hbase4:45019] zookeeper.ZKUtil(162): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39207,1676264481224 2023-02-13 05:01:21,565 DEBUG [RS:2;jenkins-hbase4:42641] zookeeper.ZKUtil(162): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42641,1676264481283 2023-02-13 05:01:21,565 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 05:01:21,566 DEBUG [RS:0;jenkins-hbase4:39207] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 05:01:21,566 DEBUG [RS:2;jenkins-hbase4:42641] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 05:01:21,566 INFO [RS:0;jenkins-hbase4:39207] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 05:01:21,566 INFO [RS:2;jenkins-hbase4:42641] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 05:01:21,567 DEBUG [RS:1;jenkins-hbase4:45019] zookeeper.ZKUtil(162): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42641,1676264481283 2023-02-13 05:01:21,567 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 05:01:21,569 INFO [RS:0;jenkins-hbase4:39207] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 05:01:21,569 DEBUG [RS:1;jenkins-hbase4:45019] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 05:01:21,571 INFO [RS:1;jenkins-hbase4:45019] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 05:01:21,576 INFO [RS:0;jenkins-hbase4:39207] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 05:01:21,576 INFO [RS:1;jenkins-hbase4:45019] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 05:01:21,576 INFO [RS:0;jenkins-hbase4:39207] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,576 INFO [RS:2;jenkins-hbase4:42641] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 05:01:21,577 INFO [RS:2;jenkins-hbase4:42641] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 05:01:21,577 INFO [RS:0;jenkins-hbase4:39207] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 05:01:21,577 INFO [RS:2;jenkins-hbase4:42641] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,578 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 05:01:21,578 INFO [RS:1;jenkins-hbase4:45019] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 05:01:21,578 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,579 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71193351, jitterRate=0.06086359918117523}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 05:01:21,579 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676264481529Initializing all the Stores at 1676264481530 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676264481530Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264481531 (+1 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676264481531Cleaning up temporary data from old regions at 1676264481564 (+33 ms)Cleaning up detritus from prior splits at 1676264481564Region opened successfully at 1676264481579 (+15 ms) 2023-02-13 05:01:21,580 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 05:01:21,580 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 05:01:21,580 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 05:01:21,580 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 05:01:21,580 INFO [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 05:01:21,580 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 05:01:21,580 INFO [RS:0;jenkins-hbase4:39207] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,580 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 05:01:21,582 DEBUG [RS:0;jenkins-hbase4:39207] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,582 DEBUG [RS:0;jenkins-hbase4:39207] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,582 DEBUG [RS:0;jenkins-hbase4:39207] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,582 DEBUG [RS:0;jenkins-hbase4:39207] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,582 DEBUG [RS:0;jenkins-hbase4:39207] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,582 DEBUG [RS:0;jenkins-hbase4:39207] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 05:01:21,582 DEBUG [RS:0;jenkins-hbase4:39207] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,582 DEBUG [RS:0;jenkins-hbase4:39207] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,582 DEBUG [RS:0;jenkins-hbase4:39207] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,582 DEBUG [RS:0;jenkins-hbase4:39207] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,588 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 05:01:21,589 INFO [RS:0;jenkins-hbase4:39207] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,589 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676264481580Disabling compacts and flushes for region at 1676264481580Disabling writes for close at 1676264481580Writing region close event to WAL at 1676264481588 (+8 ms)Closed at 1676264481588 2023-02-13 05:01:21,589 INFO [RS:0;jenkins-hbase4:39207] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,588 INFO [RS:2;jenkins-hbase4:42641] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,589 INFO [RS:0;jenkins-hbase4:39207] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,589 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,589 DEBUG [RS:2;jenkins-hbase4:42641] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,589 DEBUG [RS:2;jenkins-hbase4:42641] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,589 DEBUG [RS:2;jenkins-hbase4:42641] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,589 DEBUG [RS:2;jenkins-hbase4:42641] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,589 DEBUG [RS:2;jenkins-hbase4:42641] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,590 DEBUG [RS:2;jenkins-hbase4:42641] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 05:01:21,590 DEBUG [RS:2;jenkins-hbase4:42641] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,590 DEBUG [RS:2;jenkins-hbase4:42641] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,590 DEBUG [RS:2;jenkins-hbase4:42641] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,590 DEBUG [RS:2;jenkins-hbase4:42641] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,591 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 05:01:21,591 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-13 05:01:21,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 05:01:21,591 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,592 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,592 INFO [RS:2;jenkins-hbase4:42641] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,592 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,593 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,592 INFO [RS:2;jenkins-hbase4:42641] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,593 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,593 INFO [RS:2;jenkins-hbase4:42641] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,593 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 05:01:21,593 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,593 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,593 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,593 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 05:01:21,597 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,597 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,597 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 05:01:21,597 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,605 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 05:01:21,609 INFO [RS:0;jenkins-hbase4:39207] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 05:01:21,609 INFO [RS:0;jenkins-hbase4:39207] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39207,1676264481224-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,612 INFO [RS:1;jenkins-hbase4:45019] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 05:01:21,612 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45019,1676264481254-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,612 INFO [RS:2;jenkins-hbase4:42641] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 05:01:21,612 INFO [RS:2;jenkins-hbase4:42641] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42641,1676264481283-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:21,621 INFO [RS:0;jenkins-hbase4:39207] regionserver.Replication(203): jenkins-hbase4.apache.org,39207,1676264481224 started 2023-02-13 05:01:21,621 INFO [RS:0;jenkins-hbase4:39207] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,39207,1676264481224, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:39207, sessionid=0x1004c0e0b390001 2023-02-13 05:01:21,621 DEBUG [RS:0;jenkins-hbase4:39207] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 05:01:21,621 DEBUG [RS:0;jenkins-hbase4:39207] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,39207,1676264481224 2023-02-13 05:01:21,621 DEBUG [RS:0;jenkins-hbase4:39207] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39207,1676264481224' 2023-02-13 05:01:21,621 DEBUG [RS:0;jenkins-hbase4:39207] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 05:01:21,622 DEBUG [RS:0;jenkins-hbase4:39207] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 05:01:21,622 DEBUG [RS:0;jenkins-hbase4:39207] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 05:01:21,622 DEBUG [RS:0;jenkins-hbase4:39207] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 05:01:21,622 DEBUG [RS:0;jenkins-hbase4:39207] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,39207,1676264481224 2023-02-13 05:01:21,622 DEBUG [RS:0;jenkins-hbase4:39207] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39207,1676264481224' 2023-02-13 05:01:21,622 DEBUG [RS:0;jenkins-hbase4:39207] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 05:01:21,623 DEBUG [RS:0;jenkins-hbase4:39207] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 05:01:21,623 DEBUG [RS:0;jenkins-hbase4:39207] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 05:01:21,623 INFO [RS:0;jenkins-hbase4:39207] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 05:01:21,623 INFO [RS:0;jenkins-hbase4:39207] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 05:01:21,627 INFO [RS:1;jenkins-hbase4:45019] regionserver.Replication(203): jenkins-hbase4.apache.org,45019,1676264481254 started 2023-02-13 05:01:21,627 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,45019,1676264481254, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45019, sessionid=0x1004c0e0b390002 2023-02-13 05:01:21,627 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 05:01:21,627 DEBUG [RS:1;jenkins-hbase4:45019] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:21,627 DEBUG [RS:1;jenkins-hbase4:45019] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45019,1676264481254' 2023-02-13 05:01:21,627 DEBUG [RS:1;jenkins-hbase4:45019] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 05:01:21,627 DEBUG [RS:1;jenkins-hbase4:45019] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 05:01:21,628 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 05:01:21,628 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 05:01:21,628 DEBUG [RS:1;jenkins-hbase4:45019] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:21,628 DEBUG [RS:1;jenkins-hbase4:45019] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45019,1676264481254' 2023-02-13 05:01:21,628 DEBUG [RS:1;jenkins-hbase4:45019] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 05:01:21,628 DEBUG [RS:1;jenkins-hbase4:45019] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 05:01:21,629 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 05:01:21,629 INFO [RS:1;jenkins-hbase4:45019] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 05:01:21,629 INFO [RS:1;jenkins-hbase4:45019] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 05:01:21,632 INFO [RS:2;jenkins-hbase4:42641] regionserver.Replication(203): jenkins-hbase4.apache.org,42641,1676264481283 started 2023-02-13 05:01:21,632 INFO [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,42641,1676264481283, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42641, sessionid=0x1004c0e0b390003 2023-02-13 05:01:21,632 DEBUG [RS:2;jenkins-hbase4:42641] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 05:01:21,632 DEBUG [RS:2;jenkins-hbase4:42641] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42641,1676264481283 2023-02-13 05:01:21,632 DEBUG [RS:2;jenkins-hbase4:42641] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42641,1676264481283' 2023-02-13 05:01:21,632 DEBUG [RS:2;jenkins-hbase4:42641] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 05:01:21,632 DEBUG [RS:2;jenkins-hbase4:42641] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 05:01:21,633 DEBUG [RS:2;jenkins-hbase4:42641] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 05:01:21,633 DEBUG [RS:2;jenkins-hbase4:42641] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 05:01:21,633 DEBUG [RS:2;jenkins-hbase4:42641] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42641,1676264481283 2023-02-13 05:01:21,633 DEBUG [RS:2;jenkins-hbase4:42641] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42641,1676264481283' 2023-02-13 05:01:21,633 DEBUG [RS:2;jenkins-hbase4:42641] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 05:01:21,633 DEBUG [RS:2;jenkins-hbase4:42641] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 05:01:21,634 DEBUG [RS:2;jenkins-hbase4:42641] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 05:01:21,634 INFO [RS:2;jenkins-hbase4:42641] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 05:01:21,634 INFO [RS:2;jenkins-hbase4:42641] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 05:01:21,726 INFO [RS:0;jenkins-hbase4:39207] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39207%2C1676264481224, suffix=, logDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/WALs/jenkins-hbase4.apache.org,39207,1676264481224, archiveDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/oldWALs, maxLogs=32 2023-02-13 05:01:21,731 INFO [RS:1;jenkins-hbase4:45019] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45019%2C1676264481254, suffix=, logDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/WALs/jenkins-hbase4.apache.org,45019,1676264481254, archiveDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/oldWALs, maxLogs=32 2023-02-13 05:01:21,736 INFO [RS:2;jenkins-hbase4:42641] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42641%2C1676264481283, suffix=, logDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/WALs/jenkins-hbase4.apache.org,42641,1676264481283, archiveDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/oldWALs, maxLogs=32 2023-02-13 05:01:21,747 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43243,DS-19598a71-fd50-4434-9250-49a2527e4f19,DISK] 2023-02-13 05:01:21,747 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36757,DS-8cd3cfbb-f10f-4864-be3a-1bc6b43884e9,DISK] 2023-02-13 05:01:21,748 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43865,DS-d844d497-cdc4-4905-a59b-c5a65740ae3c,DISK] 2023-02-13 05:01:21,755 DEBUG [jenkins-hbase4:37449] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-13 05:01:21,755 DEBUG [jenkins-hbase4:37449] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 05:01:21,755 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43865,DS-d844d497-cdc4-4905-a59b-c5a65740ae3c,DISK] 2023-02-13 05:01:21,756 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43243,DS-19598a71-fd50-4434-9250-49a2527e4f19,DISK] 2023-02-13 05:01:21,756 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36757,DS-8cd3cfbb-f10f-4864-be3a-1bc6b43884e9,DISK] 2023-02-13 05:01:21,756 INFO [RS:0;jenkins-hbase4:39207] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/WALs/jenkins-hbase4.apache.org,39207,1676264481224/jenkins-hbase4.apache.org%2C39207%2C1676264481224.1676264481727 2023-02-13 05:01:21,757 DEBUG [RS:0;jenkins-hbase4:39207] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43865,DS-d844d497-cdc4-4905-a59b-c5a65740ae3c,DISK], DatanodeInfoWithStorage[127.0.0.1:43243,DS-19598a71-fd50-4434-9250-49a2527e4f19,DISK], DatanodeInfoWithStorage[127.0.0.1:36757,DS-8cd3cfbb-f10f-4864-be3a-1bc6b43884e9,DISK]] 2023-02-13 05:01:21,758 DEBUG [jenkins-hbase4:37449] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 05:01:21,758 DEBUG [jenkins-hbase4:37449] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 05:01:21,758 DEBUG [jenkins-hbase4:37449] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 05:01:21,758 DEBUG [jenkins-hbase4:37449] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 05:01:21,758 INFO [RS:1;jenkins-hbase4:45019] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/WALs/jenkins-hbase4.apache.org,45019,1676264481254/jenkins-hbase4.apache.org%2C45019%2C1676264481254.1676264481732 2023-02-13 05:01:21,759 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42641,1676264481283, state=OPENING 2023-02-13 05:01:21,760 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-13 05:01:21,761 DEBUG [RS:1;jenkins-hbase4:45019] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43865,DS-d844d497-cdc4-4905-a59b-c5a65740ae3c,DISK], DatanodeInfoWithStorage[127.0.0.1:36757,DS-8cd3cfbb-f10f-4864-be3a-1bc6b43884e9,DISK], DatanodeInfoWithStorage[127.0.0.1:43243,DS-19598a71-fd50-4434-9250-49a2527e4f19,DISK]] 2023-02-13 05:01:21,765 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:21,765 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43865,DS-d844d497-cdc4-4905-a59b-c5a65740ae3c,DISK] 2023-02-13 05:01:21,765 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43243,DS-19598a71-fd50-4434-9250-49a2527e4f19,DISK] 2023-02-13 05:01:21,765 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36757,DS-8cd3cfbb-f10f-4864-be3a-1bc6b43884e9,DISK] 2023-02-13 05:01:21,766 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 05:01:21,766 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42641,1676264481283}] 2023-02-13 05:01:21,769 INFO [RS:2;jenkins-hbase4:42641] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/WALs/jenkins-hbase4.apache.org,42641,1676264481283/jenkins-hbase4.apache.org%2C42641%2C1676264481283.1676264481737 2023-02-13 05:01:21,769 DEBUG [RS:2;jenkins-hbase4:42641] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43865,DS-d844d497-cdc4-4905-a59b-c5a65740ae3c,DISK], DatanodeInfoWithStorage[127.0.0.1:36757,DS-8cd3cfbb-f10f-4864-be3a-1bc6b43884e9,DISK], DatanodeInfoWithStorage[127.0.0.1:43243,DS-19598a71-fd50-4434-9250-49a2527e4f19,DISK]] 2023-02-13 05:01:21,921 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42641,1676264481283 2023-02-13 05:01:21,921 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 05:01:21,922 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52014, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 05:01:21,927 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 05:01:21,927 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 05:01:21,929 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42641%2C1676264481283.meta, suffix=.meta, logDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/WALs/jenkins-hbase4.apache.org,42641,1676264481283, archiveDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/oldWALs, maxLogs=32 2023-02-13 05:01:21,944 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43865,DS-d844d497-cdc4-4905-a59b-c5a65740ae3c,DISK] 2023-02-13 05:01:21,945 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36757,DS-8cd3cfbb-f10f-4864-be3a-1bc6b43884e9,DISK] 2023-02-13 05:01:21,945 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43243,DS-19598a71-fd50-4434-9250-49a2527e4f19,DISK] 2023-02-13 05:01:21,947 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/WALs/jenkins-hbase4.apache.org,42641,1676264481283/jenkins-hbase4.apache.org%2C42641%2C1676264481283.meta.1676264481930.meta 2023-02-13 05:01:21,948 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43865,DS-d844d497-cdc4-4905-a59b-c5a65740ae3c,DISK], DatanodeInfoWithStorage[127.0.0.1:36757,DS-8cd3cfbb-f10f-4864-be3a-1bc6b43884e9,DISK], DatanodeInfoWithStorage[127.0.0.1:43243,DS-19598a71-fd50-4434-9250-49a2527e4f19,DISK]] 2023-02-13 05:01:21,948 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 05:01:21,948 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 05:01:21,948 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 05:01:21,950 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 05:01:21,950 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 05:01:21,950 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 05:01:21,951 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 05:01:21,951 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 05:01:21,952 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 05:01:21,954 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/info 2023-02-13 05:01:21,954 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/info 2023-02-13 05:01:21,954 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 05:01:21,955 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:21,955 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 05:01:21,956 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/rep_barrier 2023-02-13 05:01:21,956 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/rep_barrier 2023-02-13 05:01:21,956 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 05:01:21,957 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:21,957 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 05:01:21,958 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/table 2023-02-13 05:01:21,958 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/table 2023-02-13 05:01:21,958 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 05:01:21,959 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:21,960 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740 2023-02-13 05:01:21,962 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740 2023-02-13 05:01:21,966 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 05:01:21,968 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 05:01:21,969 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62420567, jitterRate=-0.06986106932163239}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 05:01:21,969 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676264481951Writing region info on filesystem at 1676264481951Initializing all the Stores at 1676264481952 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676264481952Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264481952Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676264481952Cleaning up temporary data from old regions at 1676264481963 (+11 ms)Cleaning up detritus from prior splits at 1676264481965 (+2 ms)Running coprocessor post-open hooks at 1676264481969 (+4 ms)Region opened successfully at 1676264481969 2023-02-13 05:01:21,971 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676264481921 2023-02-13 05:01:21,975 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 05:01:21,976 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 05:01:21,976 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42641,1676264481283, state=OPEN 2023-02-13 05:01:21,979 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 05:01:21,980 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 05:01:21,982 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-13 05:01:21,983 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42641,1676264481283 in 213 msec 2023-02-13 05:01:21,986 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-13 05:01:21,986 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 392 msec 2023-02-13 05:01:21,988 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 512 msec 2023-02-13 05:01:21,988 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676264481300, completionTime=-1 2023-02-13 05:01:21,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-13 05:01:21,989 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 05:01:21,992 DEBUG [hconnection-0x4ea51a29-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 05:01:21,994 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52026, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 05:01:21,995 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 05:01:21,995 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264541995 2023-02-13 05:01:21,995 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264601995 2023-02-13 05:01:21,995 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-13 05:01:22,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37449,1676264481173-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:22,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37449,1676264481173-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:22,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37449,1676264481173-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:22,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:37449, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:22,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 05:01:22,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-13 05:01:22,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 05:01:22,004 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-13 05:01:22,004 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-13 05:01:22,006 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 05:01:22,007 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 05:01:22,009 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/.tmp/data/hbase/namespace/a30295a1dee75698f07fae4ffca01121 2023-02-13 05:01:22,010 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/.tmp/data/hbase/namespace/a30295a1dee75698f07fae4ffca01121 empty. 2023-02-13 05:01:22,010 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/.tmp/data/hbase/namespace/a30295a1dee75698f07fae4ffca01121 2023-02-13 05:01:22,010 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-13 05:01:22,027 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-13 05:01:22,029 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => a30295a1dee75698f07fae4ffca01121, NAME => 'hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/.tmp 2023-02-13 05:01:22,044 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 05:01:22,044 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing a30295a1dee75698f07fae4ffca01121, disabling compactions & flushes 2023-02-13 05:01:22,045 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121. 2023-02-13 05:01:22,045 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121. 2023-02-13 05:01:22,045 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121. after waiting 0 ms 2023-02-13 05:01:22,045 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121. 2023-02-13 05:01:22,045 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121. 2023-02-13 05:01:22,045 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for a30295a1dee75698f07fae4ffca01121: Waiting for close lock at 1676264482044Disabling compacts and flushes for region at 1676264482044Disabling writes for close at 1676264482045 (+1 ms)Writing region close event to WAL at 1676264482045Closed at 1676264482045 2023-02-13 05:01:22,048 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 05:01:22,050 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676264482049"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264482049"}]},"ts":"1676264482049"} 2023-02-13 05:01:22,052 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 05:01:22,054 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 05:01:22,054 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264482054"}]},"ts":"1676264482054"} 2023-02-13 05:01:22,056 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-13 05:01:22,060 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 05:01:22,061 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 05:01:22,061 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 05:01:22,061 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 05:01:22,061 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 05:01:22,062 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=a30295a1dee75698f07fae4ffca01121, ASSIGN}] 2023-02-13 05:01:22,064 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=a30295a1dee75698f07fae4ffca01121, ASSIGN 2023-02-13 05:01:22,065 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=a30295a1dee75698f07fae4ffca01121, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45019,1676264481254; forceNewPlan=false, retain=false 2023-02-13 05:01:22,215 INFO [jenkins-hbase4:37449] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 05:01:22,216 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=a30295a1dee75698f07fae4ffca01121, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:22,217 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676264482216"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264482216"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264482216"}]},"ts":"1676264482216"} 2023-02-13 05:01:22,219 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure a30295a1dee75698f07fae4ffca01121, server=jenkins-hbase4.apache.org,45019,1676264481254}] 2023-02-13 05:01:22,373 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:22,373 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 05:01:22,375 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60502, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 05:01:22,379 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121. 2023-02-13 05:01:22,380 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a30295a1dee75698f07fae4ffca01121, NAME => 'hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121.', STARTKEY => '', ENDKEY => ''} 2023-02-13 05:01:22,380 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace a30295a1dee75698f07fae4ffca01121 2023-02-13 05:01:22,380 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 05:01:22,380 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a30295a1dee75698f07fae4ffca01121 2023-02-13 05:01:22,380 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a30295a1dee75698f07fae4ffca01121 2023-02-13 05:01:22,381 INFO [StoreOpener-a30295a1dee75698f07fae4ffca01121-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region a30295a1dee75698f07fae4ffca01121 2023-02-13 05:01:22,383 DEBUG [StoreOpener-a30295a1dee75698f07fae4ffca01121-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/namespace/a30295a1dee75698f07fae4ffca01121/info 2023-02-13 05:01:22,383 DEBUG [StoreOpener-a30295a1dee75698f07fae4ffca01121-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/namespace/a30295a1dee75698f07fae4ffca01121/info 2023-02-13 05:01:22,384 INFO [StoreOpener-a30295a1dee75698f07fae4ffca01121-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a30295a1dee75698f07fae4ffca01121 columnFamilyName info 2023-02-13 05:01:22,385 INFO [StoreOpener-a30295a1dee75698f07fae4ffca01121-1] regionserver.HStore(310): Store=a30295a1dee75698f07fae4ffca01121/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 05:01:22,386 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/namespace/a30295a1dee75698f07fae4ffca01121 2023-02-13 05:01:22,387 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/namespace/a30295a1dee75698f07fae4ffca01121 2023-02-13 05:01:22,391 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a30295a1dee75698f07fae4ffca01121 2023-02-13 05:01:22,395 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/namespace/a30295a1dee75698f07fae4ffca01121/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 05:01:22,395 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a30295a1dee75698f07fae4ffca01121; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63037531, jitterRate=-0.06066758930683136}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 05:01:22,396 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a30295a1dee75698f07fae4ffca01121: Running coprocessor pre-open hook at 1676264482380Writing region info on filesystem at 1676264482380Initializing all the Stores at 1676264482381 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676264482381Cleaning up temporary data from old regions at 1676264482388 (+7 ms)Cleaning up detritus from prior splits at 1676264482388Running coprocessor post-open hooks at 1676264482396 (+8 ms)Region opened successfully at 1676264482396 2023-02-13 05:01:22,397 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121., pid=6, masterSystemTime=1676264482373 2023-02-13 05:01:22,400 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121. 2023-02-13 05:01:22,401 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121. 2023-02-13 05:01:22,402 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=a30295a1dee75698f07fae4ffca01121, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:22,402 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676264482401"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264482401"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264482401"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264482401"}]},"ts":"1676264482401"} 2023-02-13 05:01:22,408 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-13 05:01:22,408 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure a30295a1dee75698f07fae4ffca01121, server=jenkins-hbase4.apache.org,45019,1676264481254 in 186 msec 2023-02-13 05:01:22,411 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-13 05:01:22,412 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=a30295a1dee75698f07fae4ffca01121, ASSIGN in 346 msec 2023-02-13 05:01:22,413 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 05:01:22,413 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264482413"}]},"ts":"1676264482413"} 2023-02-13 05:01:22,415 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-13 05:01:22,418 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 05:01:22,420 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 415 msec 2023-02-13 05:01:22,505 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-13 05:01:22,507 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-13 05:01:22,507 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:22,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 05:01:22,512 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60518, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 05:01:22,514 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-13 05:01:22,525 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 05:01:22,532 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 18 msec 2023-02-13 05:01:22,537 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-13 05:01:22,545 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 05:01:22,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 11 msec 2023-02-13 05:01:22,562 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 05:01:22,564 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 05:01:22,564 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 1.258sec 2023-02-13 05:01:22,564 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 05:01:22,564 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 05:01:22,564 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 05:01:22,565 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37449,1676264481173-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 05:01:22,565 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37449,1676264481173-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 05:01:22,567 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 05:01:22,599 DEBUG [Listener at localhost/35553] zookeeper.ReadOnlyZKClient(139): Connect 0x17227ed0 to 127.0.0.1:61432 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 05:01:22,604 DEBUG [Listener at localhost/35553] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4515972f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 05:01:22,606 DEBUG [hconnection-0x7284ed35-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 05:01:22,608 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52040, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 05:01:22,610 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,37449,1676264481173 2023-02-13 05:01:22,610 DEBUG [Listener at localhost/35553] zookeeper.ReadOnlyZKClient(139): Connect 0x7ac40a55 to 127.0.0.1:61432 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 05:01:22,618 DEBUG [ReadOnlyZKClient-127.0.0.1:61432@0x7ac40a55] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7100a225, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 05:01:22,619 DEBUG [Listener at localhost/35553] client.ConnectionUtils(586): Start fetching master stub from registry 2023-02-13 05:01:22,620 DEBUG [ReadOnlyZKClient-127.0.0.1:61432@0x7ac40a55] client.AsyncConnectionImpl(289): The fetched master address is jenkins-hbase4.apache.org,37449,1676264481173 2023-02-13 05:01:22,620 DEBUG [ReadOnlyZKClient-127.0.0.1:61432@0x7ac40a55] client.ConnectionUtils(594): The fetched master stub is org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$Stub@753f28dd 2023-02-13 05:01:22,624 DEBUG [ReadOnlyZKClient-127.0.0.1:61432@0x7ac40a55] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 05:01:22,625 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44744, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 05:01:22,626 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37449] master.MasterRpcServices(1560): Client=jenkins//172.31.14.131 shutdown 2023-02-13 05:01:22,626 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37449] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,37449,1676264481173 2023-02-13 05:01:22,627 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 05:01:22,627 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 05:01:22,627 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37449] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 05:01:22,627 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 05:01:22,627 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:39207-0x1004c0e0b390001, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 05:01:22,628 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:22,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 05:01:22,629 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37449] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x139f1282 to 127.0.0.1:61432 2023-02-13 05:01:22,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 05:01:22,629 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37449] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:22,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 05:01:22,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39207-0x1004c0e0b390001, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 05:01:22,670 INFO [RS:0;jenkins-hbase4:39207] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,39207,1676264481224' ***** 2023-02-13 05:01:22,670 INFO [RS:0;jenkins-hbase4:39207] regionserver.HRegionServer(2310): STOPPED: Exiting; cluster shutdown set and not carrying any regions 2023-02-13 05:01:22,671 INFO [RS:0;jenkins-hbase4:39207] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 05:01:22,672 INFO [RS:0;jenkins-hbase4:39207] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 05:01:22,672 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 05:01:22,672 INFO [RS:0;jenkins-hbase4:39207] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 05:01:22,672 INFO [RS:0;jenkins-hbase4:39207] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,39207,1676264481224 2023-02-13 05:01:22,672 DEBUG [RS:0;jenkins-hbase4:39207] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x03b6dd4f to 127.0.0.1:61432 2023-02-13 05:01:22,673 DEBUG [RS:0;jenkins-hbase4:39207] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:22,673 INFO [RS:0;jenkins-hbase4:39207] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,39207,1676264481224; all regions closed. 2023-02-13 05:01:22,677 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1065): Closing user regions 2023-02-13 05:01:22,677 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(3304): Received CLOSE for a30295a1dee75698f07fae4ffca01121 2023-02-13 05:01:22,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a30295a1dee75698f07fae4ffca01121, disabling compactions & flushes 2023-02-13 05:01:22,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121. 2023-02-13 05:01:22,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121. 2023-02-13 05:01:22,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121. after waiting 0 ms 2023-02-13 05:01:22,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121. 2023-02-13 05:01:22,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing a30295a1dee75698f07fae4ffca01121 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-13 05:01:22,681 DEBUG [RS:0;jenkins-hbase4:39207] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/oldWALs 2023-02-13 05:01:22,681 INFO [RS:0;jenkins-hbase4:39207] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39207%2C1676264481224:(num 1676264481727) 2023-02-13 05:01:22,681 DEBUG [RS:0;jenkins-hbase4:39207] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:22,681 INFO [RS:0;jenkins-hbase4:39207] regionserver.LeaseManager(133): Closed leases 2023-02-13 05:01:22,681 INFO [RS:0;jenkins-hbase4:39207] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 05:01:22,681 INFO [RS:0;jenkins-hbase4:39207] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 05:01:22,681 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 05:01:22,681 INFO [RS:0;jenkins-hbase4:39207] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 05:01:22,681 INFO [RS:0;jenkins-hbase4:39207] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 05:01:22,682 INFO [RS:0;jenkins-hbase4:39207] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39207 2023-02-13 05:01:22,682 INFO [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(1065): Closing user regions 2023-02-13 05:01:22,688 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39207,1676264481224 2023-02-13 05:01:22,688 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39207,1676264481224 2023-02-13 05:01:22,688 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 05:01:22,688 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:39207-0x1004c0e0b390001, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39207,1676264481224 2023-02-13 05:01:22,688 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 05:01:22,688 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:39207-0x1004c0e0b390001, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 05:01:22,688 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 05:01:22,692 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 05:01:22,693 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,39207,1676264481224] 2023-02-13 05:01:22,694 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39207,1676264481224; numProcessing=1 2023-02-13 05:01:22,694 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:22,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:22,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42641,1676264481283 2023-02-13 05:01:22,695 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,39207,1676264481224 already deleted, retry=false 2023-02-13 05:01:22,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42641,1676264481283 2023-02-13 05:01:22,695 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,39207,1676264481224 expired; onlineServers=2 2023-02-13 05:01:22,700 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,39207,1676264481224 znode expired, triggering replicatorRemoved event 2023-02-13 05:01:22,701 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,39207,1676264481224 znode expired, triggering replicatorRemoved event 2023-02-13 05:01:22,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:22,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:22,705 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42641,1676264481283 2023-02-13 05:01:22,705 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42641,1676264481283 2023-02-13 05:01:22,708 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/namespace/a30295a1dee75698f07fae4ffca01121/.tmp/info/99ab021a16e44a17aa116996f5b4eed5 2023-02-13 05:01:22,718 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/namespace/a30295a1dee75698f07fae4ffca01121/.tmp/info/99ab021a16e44a17aa116996f5b4eed5 as hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/namespace/a30295a1dee75698f07fae4ffca01121/info/99ab021a16e44a17aa116996f5b4eed5 2023-02-13 05:01:22,724 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/namespace/a30295a1dee75698f07fae4ffca01121/info/99ab021a16e44a17aa116996f5b4eed5, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 05:01:22,726 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for a30295a1dee75698f07fae4ffca01121 in 48ms, sequenceid=6, compaction requested=false 2023-02-13 05:01:22,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/namespace/a30295a1dee75698f07fae4ffca01121/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-13 05:01:22,735 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121. 2023-02-13 05:01:22,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a30295a1dee75698f07fae4ffca01121: Waiting for close lock at 1676264482677Running coprocessor pre-close hooks at 1676264482677Disabling compacts and flushes for region at 1676264482678 (+1 ms)Disabling writes for close at 1676264482678Obtaining lock to block concurrent updates at 1676264482678Preparing flush snapshotting stores in a30295a1dee75698f07fae4ffca01121 at 1676264482678Finished memstore snapshotting hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676264482679 (+1 ms)Flushing stores of hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121. at 1676264482679Flushing a30295a1dee75698f07fae4ffca01121/info: creating writer at 1676264482679Flushing a30295a1dee75698f07fae4ffca01121/info: appending metadata at 1676264482687 (+8 ms)Flushing a30295a1dee75698f07fae4ffca01121/info: closing flushed file at 1676264482688 (+1 ms)Flushing a30295a1dee75698f07fae4ffca01121/info: reopening flushed file at 1676264482718 (+30 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for a30295a1dee75698f07fae4ffca01121 in 48ms, sequenceid=6, compaction requested=false at 1676264482726 (+8 ms)Writing region close event to WAL at 1676264482728 (+2 ms)Running coprocessor post-close hooks at 1676264482734 (+6 ms)Closed at 1676264482735 (+1 ms) 2023-02-13 05:01:22,736 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37449] assignment.AssignmentManager(1094): RegionServer CLOSED a30295a1dee75698f07fae4ffca01121 2023-02-13 05:01:22,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676264482003.a30295a1dee75698f07fae4ffca01121. 2023-02-13 05:01:22,779 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,45019,1676264481254' ***** 2023-02-13 05:01:22,779 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(2310): STOPPED: Exiting; cluster shutdown set and not carrying any regions 2023-02-13 05:01:22,781 INFO [RS:1;jenkins-hbase4:45019] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 05:01:22,781 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 05:01:22,781 INFO [RS:1;jenkins-hbase4:45019] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 05:01:22,781 INFO [RS:1;jenkins-hbase4:45019] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 05:01:22,781 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:22,782 DEBUG [RS:1;jenkins-hbase4:45019] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x04643388 to 127.0.0.1:61432 2023-02-13 05:01:22,782 DEBUG [RS:1;jenkins-hbase4:45019] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:22,782 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,45019,1676264481254; all regions closed. 2023-02-13 05:01:22,786 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/WALs/jenkins-hbase4.apache.org,45019,1676264481254/jenkins-hbase4.apache.org%2C45019%2C1676264481254.1676264481732 not finished, retry = 0 2023-02-13 05:01:22,787 DEBUG [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(1084): Waiting on 1588230740 2023-02-13 05:01:22,802 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 05:01:22,830 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:39207-0x1004c0e0b390001, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:22,830 INFO [RS:0;jenkins-hbase4:39207] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,39207,1676264481224; zookeeper connection closed. 2023-02-13 05:01:22,830 ERROR [Listener at localhost/35553-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@442afac6 rejected from java.util.concurrent.ThreadPoolExecutor@4015669b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:22,831 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@51f5452a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@51f5452a 2023-02-13 05:01:22,831 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:39207-0x1004c0e0b390001, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:22,832 ERROR [Listener at localhost/35553-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@27b737e1 rejected from java.util.concurrent.ThreadPoolExecutor@4015669b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:22,889 INFO [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,42641,1676264481283' ***** 2023-02-13 05:01:22,889 INFO [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(2310): STOPPED: Stopped; only catalog regions remaining online 2023-02-13 05:01:22,889 INFO [RS:2;jenkins-hbase4:42641] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 05:01:22,889 DEBUG [RS:1;jenkins-hbase4:45019] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/oldWALs 2023-02-13 05:01:22,889 INFO [RS:2;jenkins-hbase4:42641] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 05:01:22,889 INFO [RS:1;jenkins-hbase4:45019] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45019%2C1676264481254:(num 1676264481732) 2023-02-13 05:01:22,890 DEBUG [RS:1;jenkins-hbase4:45019] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:22,889 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 05:01:22,890 INFO [RS:1;jenkins-hbase4:45019] regionserver.LeaseManager(133): Closed leases 2023-02-13 05:01:22,890 INFO [RS:2;jenkins-hbase4:42641] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 05:01:22,890 INFO [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,42641,1676264481283 2023-02-13 05:01:22,890 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 05:01:22,890 DEBUG [RS:2;jenkins-hbase4:42641] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x34e8d17a to 127.0.0.1:61432 2023-02-13 05:01:22,891 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 05:01:22,891 DEBUG [RS:2;jenkins-hbase4:42641] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:22,891 INFO [RS:1;jenkins-hbase4:45019] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 05:01:22,891 INFO [RS:2;jenkins-hbase4:42641] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 05:01:22,891 INFO [RS:2;jenkins-hbase4:42641] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 05:01:22,891 INFO [RS:2;jenkins-hbase4:42641] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 05:01:22,891 INFO [RS:1;jenkins-hbase4:45019] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 05:01:22,891 INFO [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 05:01:22,891 INFO [RS:1;jenkins-hbase4:45019] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 05:01:22,891 INFO [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 05:01:22,891 DEBUG [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-13 05:01:22,892 INFO [RS:1;jenkins-hbase4:45019] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45019 2023-02-13 05:01:22,892 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 05:01:22,892 DEBUG [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 05:01:22,892 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 05:01:22,893 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 05:01:22,893 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 05:01:22,893 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 05:01:22,895 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-02-13 05:01:22,896 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 05:01:22,896 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:22,896 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,45019,1676264481254 2023-02-13 05:01:22,896 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 05:01:22,896 ERROR [Listener at localhost/35553-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6b3bd37c rejected from java.util.concurrent.ThreadPoolExecutor@6f044ac3[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:22,897 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 05:01:22,897 ERROR [Listener at localhost/35553-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2e73840d rejected from java.util.concurrent.ThreadPoolExecutor@6f044ac3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:22,897 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45019,1676264481254] 2023-02-13 05:01:22,897 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45019,1676264481254; numProcessing=2 2023-02-13 05:01:22,899 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,45019,1676264481254 already deleted, retry=false 2023-02-13 05:01:22,899 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45019,1676264481254 expired; onlineServers=1 2023-02-13 05:01:22,901 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 05:01:22,914 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/.tmp/info/1743c469f1324c3dba45bcbf4d040e48 2023-02-13 05:01:22,935 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/.tmp/table/640862881ab14dee87f3a18c544bf48a 2023-02-13 05:01:22,944 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/.tmp/info/1743c469f1324c3dba45bcbf4d040e48 as hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/info/1743c469f1324c3dba45bcbf4d040e48 2023-02-13 05:01:22,950 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/info/1743c469f1324c3dba45bcbf4d040e48, entries=10, sequenceid=9, filesize=5.9 K 2023-02-13 05:01:22,952 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/.tmp/table/640862881ab14dee87f3a18c544bf48a as hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/table/640862881ab14dee87f3a18c544bf48a 2023-02-13 05:01:22,958 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/table/640862881ab14dee87f3a18c544bf48a, entries=2, sequenceid=9, filesize=4.7 K 2023-02-13 05:01:22,960 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 66ms, sequenceid=9, compaction requested=false 2023-02-13 05:01:22,966 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-02-13 05:01:22,966 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 05:01:22,966 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 05:01:22,966 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676264482892Running coprocessor pre-close hooks at 1676264482892Disabling compacts and flushes for region at 1676264482892Disabling writes for close at 1676264482893 (+1 ms)Obtaining lock to block concurrent updates at 1676264482895 (+2 ms)Preparing flush snapshotting stores in 1588230740 at 1676264482895Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=1290, getHeapSize=2912, getOffHeapSize=0, getCellsCount=12 at 1676264482896 (+1 ms)Flushing stores of hbase:meta,,1.1588230740 at 1676264482897 (+1 ms)Flushing 1588230740/info: creating writer at 1676264482897Flushing 1588230740/info: appending metadata at 1676264482901 (+4 ms)Flushing 1588230740/info: closing flushed file at 1676264482901Flushing 1588230740/table: creating writer at 1676264482922 (+21 ms)Flushing 1588230740/table: appending metadata at 1676264482925 (+3 ms)Flushing 1588230740/table: closing flushed file at 1676264482925Flushing 1588230740/info: reopening flushed file at 1676264482944 (+19 ms)Flushing 1588230740/table: reopening flushed file at 1676264482953 (+9 ms)Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 66ms, sequenceid=9, compaction requested=false at 1676264482960 (+7 ms)Writing region close event to WAL at 1676264482962 (+2 ms)Running coprocessor post-close hooks at 1676264482966 (+4 ms)Closed at 1676264482966 2023-02-13 05:01:22,966 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 05:01:23,031 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:23,031 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,45019,1676264481254; zookeeper connection closed. 2023-02-13 05:01:23,032 ERROR [Listener at localhost/35553-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@56791161 rejected from java.util.concurrent.ThreadPoolExecutor@6f044ac3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:23,032 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3984fb2d] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3984fb2d 2023-02-13 05:01:23,032 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x1004c0e0b390002, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:23,032 ERROR [Listener at localhost/35553-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@78c890ac rejected from java.util.concurrent.ThreadPoolExecutor@6f044ac3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:23,093 INFO [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,42641,1676264481283; all regions closed. 2023-02-13 05:01:23,098 DEBUG [RS:2;jenkins-hbase4:42641] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/oldWALs 2023-02-13 05:01:23,098 INFO [RS:2;jenkins-hbase4:42641] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42641%2C1676264481283.meta:.meta(num 1676264481930) 2023-02-13 05:01:23,104 DEBUG [RS:2;jenkins-hbase4:42641] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/oldWALs 2023-02-13 05:01:23,104 INFO [RS:2;jenkins-hbase4:42641] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42641%2C1676264481283:(num 1676264481737) 2023-02-13 05:01:23,104 DEBUG [RS:2;jenkins-hbase4:42641] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:23,104 INFO [RS:2;jenkins-hbase4:42641] regionserver.LeaseManager(133): Closed leases 2023-02-13 05:01:23,104 INFO [RS:2;jenkins-hbase4:42641] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 05:01:23,104 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 05:01:23,105 INFO [RS:2;jenkins-hbase4:42641] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42641 2023-02-13 05:01:23,109 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42641,1676264481283 2023-02-13 05:01:23,109 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 05:01:23,109 ERROR [Listener at localhost/35553-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@310f0f8b rejected from java.util.concurrent.ThreadPoolExecutor@35309d72[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:23,110 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42641,1676264481283] 2023-02-13 05:01:23,110 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42641,1676264481283; numProcessing=3 2023-02-13 05:01:23,111 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42641,1676264481283 already deleted, retry=false 2023-02-13 05:01:23,112 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42641,1676264481283 expired; onlineServers=0 2023-02-13 05:01:23,112 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,37449,1676264481173' ***** 2023-02-13 05:01:23,112 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-13 05:01:23,112 DEBUG [M:0;jenkins-hbase4:37449] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7adabfa2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 05:01:23,112 INFO [M:0;jenkins-hbase4:37449] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,37449,1676264481173 2023-02-13 05:01:23,112 INFO [M:0;jenkins-hbase4:37449] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,37449,1676264481173; all regions closed. 2023-02-13 05:01:23,112 DEBUG [M:0;jenkins-hbase4:37449] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:23,112 DEBUG [M:0;jenkins-hbase4:37449] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 05:01:23,112 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 05:01:23,112 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676264481483] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676264481483,5,FailOnTimeoutGroup] 2023-02-13 05:01:23,112 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676264481483] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676264481483,5,FailOnTimeoutGroup] 2023-02-13 05:01:23,112 DEBUG [M:0;jenkins-hbase4:37449] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 05:01:23,114 INFO [M:0;jenkins-hbase4:37449] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 05:01:23,114 INFO [M:0;jenkins-hbase4:37449] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 05:01:23,114 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 05:01:23,114 INFO [M:0;jenkins-hbase4:37449] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-13 05:01:23,114 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 05:01:23,114 DEBUG [M:0;jenkins-hbase4:37449] master.HMaster(1502): Stopping service threads 2023-02-13 05:01:23,114 INFO [M:0;jenkins-hbase4:37449] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 05:01:23,114 ERROR [M:0;jenkins-hbase4:37449] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-2,5,PEWorkerGroup] 2023-02-13 05:01:23,115 INFO [M:0;jenkins-hbase4:37449] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 05:01:23,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 05:01:23,115 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 05:01:23,115 DEBUG [M:0;jenkins-hbase4:37449] zookeeper.ZKUtil(398): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 05:01:23,115 WARN [M:0;jenkins-hbase4:37449] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 05:01:23,115 INFO [M:0;jenkins-hbase4:37449] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 05:01:23,115 INFO [M:0;jenkins-hbase4:37449] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 05:01:23,116 DEBUG [M:0;jenkins-hbase4:37449] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 05:01:23,116 INFO [M:0;jenkins-hbase4:37449] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:23,116 DEBUG [M:0;jenkins-hbase4:37449] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:23,116 DEBUG [M:0;jenkins-hbase4:37449] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 05:01:23,116 DEBUG [M:0;jenkins-hbase4:37449] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:23,116 INFO [M:0;jenkins-hbase4:37449] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.07 KB heapSize=29.55 KB 2023-02-13 05:01:23,132 INFO [Listener at localhost/35553] client.AsyncConnectionImpl(207): Connection has been closed by Listener at localhost/35553. 2023-02-13 05:01:23,132 DEBUG [Listener at localhost/35553] client.AsyncConnectionImpl(232): Call stack: at java.lang.Thread.getStackTrace(Thread.java:1564) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:209) at org.apache.hbase.thirdparty.com.google.common.io.Closeables.close(Closeables.java:79) at org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2.tearDown(TestAsyncClusterAdminApi2.java:75) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2023-02-13 05:01:23,132 DEBUG [Listener at localhost/35553] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:23,133 DEBUG [Listener at localhost/35553] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7ac40a55 to 127.0.0.1:61432 2023-02-13 05:01:23,133 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-13 05:01:23,133 DEBUG [Listener at localhost/35553] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x17227ed0 to 127.0.0.1:61432 2023-02-13 05:01:23,133 INFO [M:0;jenkins-hbase4:37449] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.07 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b71b8ddad80c45789f2300ce5cbd602d 2023-02-13 05:01:23,134 DEBUG [Listener at localhost/35553] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 05:01:23,135 DEBUG [Listener at localhost/35553] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-13 05:01:23,142 DEBUG [M:0;jenkins-hbase4:37449] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b71b8ddad80c45789f2300ce5cbd602d as hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b71b8ddad80c45789f2300ce5cbd602d 2023-02-13 05:01:23,148 INFO [M:0;jenkins-hbase4:37449] regionserver.HStore(1080): Added hdfs://localhost:38635/user/jenkins/test-data/48dbde3e-02c3-5916-9dc1-36e3f2f47e8a/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b71b8ddad80c45789f2300ce5cbd602d, entries=8, sequenceid=66, filesize=6.3 K 2023-02-13 05:01:23,149 INFO [M:0;jenkins-hbase4:37449] regionserver.HRegion(2947): Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 33ms, sequenceid=66, compaction requested=false 2023-02-13 05:01:23,150 INFO [M:0;jenkins-hbase4:37449] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 05:01:23,150 DEBUG [M:0;jenkins-hbase4:37449] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676264483116Disabling compacts and flushes for region at 1676264483116Disabling writes for close at 1676264483116Obtaining lock to block concurrent updates at 1676264483116Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676264483116Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=24646, getHeapSize=30248, getOffHeapSize=0, getCellsCount=71 at 1676264483116Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676264483117 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676264483118 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676264483121 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676264483121Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676264483143 (+22 ms)Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 33ms, sequenceid=66, compaction requested=false at 1676264483149 (+6 ms)Writing region close event to WAL at 1676264483150 (+1 ms)Closed at 1676264483150 2023-02-13 05:01:23,153 INFO [M:0;jenkins-hbase4:37449] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 05:01:23,153 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 05:01:23,154 INFO [M:0;jenkins-hbase4:37449] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37449 2023-02-13 05:01:23,156 DEBUG [M:0;jenkins-hbase4:37449] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,37449,1676264481173 already deleted, retry=false 2023-02-13 05:01:23,317 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-02-13 05:01:23,331 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:23,331 INFO [M:0;jenkins-hbase4:37449] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,37449,1676264481173; zookeeper connection closed. 2023-02-13 05:01:23,331 ERROR [Listener at localhost/35553-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@707c6dad rejected from java.util.concurrent.ThreadPoolExecutor@7443e77a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 28] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:23,333 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): master:37449-0x1004c0e0b390000, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:23,333 ERROR [Listener at localhost/35553-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@13b86668 rejected from java.util.concurrent.ThreadPoolExecutor@7443e77a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 28] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:23,632 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:23,632 INFO [RS:2;jenkins-hbase4:42641] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,42641,1676264481283; zookeeper connection closed. 2023-02-13 05:01:23,633 ERROR [Listener at localhost/35553-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@8328552 rejected from java.util.concurrent.ThreadPoolExecutor@35309d72[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:23,633 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@73a75345] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@73a75345 2023-02-13 05:01:23,633 DEBUG [Listener at localhost/35553-EventThread] zookeeper.ZKWatcher(600): regionserver:42641-0x1004c0e0b390003, quorum=127.0.0.1:61432, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 05:01:23,633 INFO [Listener at localhost/35553] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-13 05:01:23,633 ERROR [Listener at localhost/35553-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@70626ec6 rejected from java.util.concurrent.ThreadPoolExecutor@35309d72[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 05:01:23,634 WARN [Listener at localhost/35553] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 05:01:23,638 INFO [Listener at localhost/35553] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 05:01:23,742 WARN [BP-51181900-172.31.14.131-1676264480432 heartbeating to localhost/127.0.0.1:38635] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 05:01:23,742 WARN [BP-51181900-172.31.14.131-1676264480432 heartbeating to localhost/127.0.0.1:38635] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-51181900-172.31.14.131-1676264480432 (Datanode Uuid 2d27b5c6-3da9-420d-9bfa-721f4684c8c2) service to localhost/127.0.0.1:38635 2023-02-13 05:01:23,743 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/cluster_bcbf8140-8def-93b2-7aef-cc1ac3cf1091/dfs/data/data5/current/BP-51181900-172.31.14.131-1676264480432] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 05:01:23,743 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/cluster_bcbf8140-8def-93b2-7aef-cc1ac3cf1091/dfs/data/data6/current/BP-51181900-172.31.14.131-1676264480432] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 05:01:23,744 WARN [Listener at localhost/35553] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 05:01:23,747 INFO [Listener at localhost/35553] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 05:01:23,851 WARN [BP-51181900-172.31.14.131-1676264480432 heartbeating to localhost/127.0.0.1:38635] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 05:01:23,852 WARN [BP-51181900-172.31.14.131-1676264480432 heartbeating to localhost/127.0.0.1:38635] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-51181900-172.31.14.131-1676264480432 (Datanode Uuid 762c31f2-9324-439e-94b6-570be9285088) service to localhost/127.0.0.1:38635 2023-02-13 05:01:23,852 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/cluster_bcbf8140-8def-93b2-7aef-cc1ac3cf1091/dfs/data/data3/current/BP-51181900-172.31.14.131-1676264480432] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 05:01:23,853 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/cluster_bcbf8140-8def-93b2-7aef-cc1ac3cf1091/dfs/data/data4/current/BP-51181900-172.31.14.131-1676264480432] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 05:01:23,854 WARN [Listener at localhost/35553] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 05:01:23,860 INFO [Listener at localhost/35553] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 05:01:23,963 WARN [BP-51181900-172.31.14.131-1676264480432 heartbeating to localhost/127.0.0.1:38635] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 05:01:23,963 WARN [BP-51181900-172.31.14.131-1676264480432 heartbeating to localhost/127.0.0.1:38635] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-51181900-172.31.14.131-1676264480432 (Datanode Uuid 98cfa102-ef1a-4f17-b112-c1df27aaca41) service to localhost/127.0.0.1:38635 2023-02-13 05:01:23,964 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/cluster_bcbf8140-8def-93b2-7aef-cc1ac3cf1091/dfs/data/data1/current/BP-51181900-172.31.14.131-1676264480432] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 05:01:23,964 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d262404e-0a00-e556-7b38-96b79c3a8a7f/cluster_bcbf8140-8def-93b2-7aef-cc1ac3cf1091/dfs/data/data2/current/BP-51181900-172.31.14.131-1676264480432] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 05:01:23,977 INFO [Listener at localhost/35553] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 05:01:24,092 INFO [Listener at localhost/35553] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-13 05:01:24,118 INFO [Listener at localhost/35553] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-02-13 05:01:24,129 INFO [Listener at localhost/35553] hbase.ResourceChecker(175): after: client.TestAsyncClusterAdminApi2#testShutdown Thread=107 (was 76) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager$NodeFailoverWorker.run(ReplicationSourceManager.java:703) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/35553 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:38635 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:38635 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1171269600) connection to localhost/127.0.0.1:38635 from jenkins.hfs.4 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-9-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1171269600) connection to localhost/127.0.0.1:38635 from jenkins.hfs.3 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:38635 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1171269600) connection to localhost/127.0.0.1:38635 from jenkins.hfs.5 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1171269600) connection to localhost/127.0.0.1:38635 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1171269600) connection to localhost/127.0.0.1:38635 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:38635 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager$NodeFailoverWorker.run(ReplicationSourceManager.java:703) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=546 (was 506) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=191 (was 190) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3546 (was 3625)