2023-02-13 21:59:42,457 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.TestRetainAssignmentOnRestart timeout: 13 mins 2023-02-13 21:59:42,459 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.TestRetainAssignmentOnRestartSplitWithoutZk timeout: 13 mins 2023-02-13 21:59:42,497 INFO [Time-limited test] hbase.ResourceChecker(147): before: master.TestRetainAssignmentOnRestartSplitWithoutZk#testRetainAssignmentOnSingleRSRestart Thread=7, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=208, ProcessCount=164, AvailableMemoryMB=7822 2023-02-13 21:59:42,713 DEBUG [Time-limited test] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c 2023-02-13 21:59:42,715 INFO [Time-limited test] master.AbstractTestRestartCluster(45): WAL splitting coordinated by zk false 2023-02-13 21:59:42,718 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-13 21:59:42,718 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/cluster_0697ba01-9152-a749-74f7-a697351678b4, deleteOnExit=true 2023-02-13 21:59:42,719 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-13 21:59:42,719 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/test.cache.data in system properties and HBase conf 2023-02-13 21:59:42,720 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/hadoop.tmp.dir in system properties and HBase conf 2023-02-13 21:59:42,720 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/hadoop.log.dir in system properties and HBase conf 2023-02-13 21:59:42,720 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-13 21:59:42,721 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-13 21:59:42,721 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-13 21:59:42,808 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-13 21:59:43,160 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-13 21:59:43,166 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-13 21:59:43,166 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-13 21:59:43,167 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-13 21:59:43,167 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 21:59:43,167 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-13 21:59:43,167 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-13 21:59:43,168 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 21:59:43,168 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 21:59:43,168 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-13 21:59:43,169 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/nfs.dump.dir in system properties and HBase conf 2023-02-13 21:59:43,169 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/java.io.tmpdir in system properties and HBase conf 2023-02-13 21:59:43,169 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 21:59:43,169 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-13 21:59:43,170 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-13 21:59:43,575 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 21:59:43,579 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 21:59:43,804 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-13 21:59:43,951 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-13 21:59:43,964 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 21:59:44,001 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 21:59:44,029 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/java.io.tmpdir/Jetty_localhost_localdomain_38715_hdfs____.8izpcg/webapp 2023-02-13 21:59:44,161 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:38715 2023-02-13 21:59:44,169 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 21:59:44,169 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 21:59:44,533 WARN [Listener at localhost.localdomain/36521] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 21:59:44,597 WARN [Listener at localhost.localdomain/36521] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 21:59:44,615 WARN [Listener at localhost.localdomain/36521] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 21:59:44,621 INFO [Listener at localhost.localdomain/36521] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 21:59:44,625 INFO [Listener at localhost.localdomain/36521] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/java.io.tmpdir/Jetty_localhost_37495_datanode____2wjexf/webapp 2023-02-13 21:59:44,716 INFO [Listener at localhost.localdomain/36521] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37495 2023-02-13 21:59:45,009 WARN [Listener at localhost.localdomain/40583] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 21:59:45,024 WARN [Listener at localhost.localdomain/40583] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 21:59:45,027 WARN [Listener at localhost.localdomain/40583] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 21:59:45,030 INFO [Listener at localhost.localdomain/40583] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 21:59:45,036 INFO [Listener at localhost.localdomain/40583] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/java.io.tmpdir/Jetty_localhost_34987_datanode____9hls9q/webapp 2023-02-13 21:59:45,133 INFO [Listener at localhost.localdomain/40583] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34987 2023-02-13 21:59:45,144 WARN [Listener at localhost.localdomain/40899] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 21:59:45,164 WARN [Listener at localhost.localdomain/40899] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 21:59:45,167 WARN [Listener at localhost.localdomain/40899] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 21:59:45,168 INFO [Listener at localhost.localdomain/40899] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 21:59:45,176 INFO [Listener at localhost.localdomain/40899] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/java.io.tmpdir/Jetty_localhost_32863_datanode____.xgeyub/webapp 2023-02-13 21:59:45,281 INFO [Listener at localhost.localdomain/40899] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:32863 2023-02-13 21:59:45,291 WARN [Listener at localhost.localdomain/34171] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 21:59:45,500 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x85ed80e70af802bb: Processing first storage report for DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92 from datanode d8f56787-7e58-4399-ae6d-6fb700e1d7bd 2023-02-13 21:59:45,501 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x85ed80e70af802bb: from storage DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92 node DatanodeRegistration(127.0.0.1:37281, datanodeUuid=d8f56787-7e58-4399-ae6d-6fb700e1d7bd, infoPort=40673, infoSecurePort=0, ipcPort=40899, storageInfo=lv=-57;cid=testClusterID;nsid=1108089026;c=1676325583640), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-13 21:59:45,501 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x59f41b61f89e1675: Processing first storage report for DS-22c25aa3-a398-4428-b14b-650d35a9a0f7 from datanode 8e39e35d-a56a-4b9c-a217-5e2392a9f1a9 2023-02-13 21:59:45,502 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x59f41b61f89e1675: from storage DS-22c25aa3-a398-4428-b14b-650d35a9a0f7 node DatanodeRegistration(127.0.0.1:40259, datanodeUuid=8e39e35d-a56a-4b9c-a217-5e2392a9f1a9, infoPort=35019, infoSecurePort=0, ipcPort=34171, storageInfo=lv=-57;cid=testClusterID;nsid=1108089026;c=1676325583640), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 21:59:45,502 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x85ed80e70af802bb: Processing first storage report for DS-3b588a51-6055-471f-b5c2-088778bacfc9 from datanode d8f56787-7e58-4399-ae6d-6fb700e1d7bd 2023-02-13 21:59:45,502 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x85ed80e70af802bb: from storage DS-3b588a51-6055-471f-b5c2-088778bacfc9 node DatanodeRegistration(127.0.0.1:37281, datanodeUuid=d8f56787-7e58-4399-ae6d-6fb700e1d7bd, infoPort=40673, infoSecurePort=0, ipcPort=40899, storageInfo=lv=-57;cid=testClusterID;nsid=1108089026;c=1676325583640), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 21:59:45,502 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x59f41b61f89e1675: Processing first storage report for DS-9db6a3ba-1516-4a8c-8c8a-3395ec17bee9 from datanode 8e39e35d-a56a-4b9c-a217-5e2392a9f1a9 2023-02-13 21:59:45,502 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x59f41b61f89e1675: from storage DS-9db6a3ba-1516-4a8c-8c8a-3395ec17bee9 node DatanodeRegistration(127.0.0.1:40259, datanodeUuid=8e39e35d-a56a-4b9c-a217-5e2392a9f1a9, infoPort=35019, infoSecurePort=0, ipcPort=34171, storageInfo=lv=-57;cid=testClusterID;nsid=1108089026;c=1676325583640), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 21:59:45,502 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x5d6cd4b00d6b20d7: Processing first storage report for DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801 from datanode fdce1170-0de6-4c94-80d6-58e0a1ca2df6 2023-02-13 21:59:45,502 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x5d6cd4b00d6b20d7: from storage DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801 node DatanodeRegistration(127.0.0.1:38109, datanodeUuid=fdce1170-0de6-4c94-80d6-58e0a1ca2df6, infoPort=33589, infoSecurePort=0, ipcPort=40583, storageInfo=lv=-57;cid=testClusterID;nsid=1108089026;c=1676325583640), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 21:59:45,503 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x5d6cd4b00d6b20d7: Processing first storage report for DS-cca4bf32-9354-4ec7-b1b9-71e129397761 from datanode fdce1170-0de6-4c94-80d6-58e0a1ca2df6 2023-02-13 21:59:45,503 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x5d6cd4b00d6b20d7: from storage DS-cca4bf32-9354-4ec7-b1b9-71e129397761 node DatanodeRegistration(127.0.0.1:38109, datanodeUuid=fdce1170-0de6-4c94-80d6-58e0a1ca2df6, infoPort=33589, infoSecurePort=0, ipcPort=40583, storageInfo=lv=-57;cid=testClusterID;nsid=1108089026;c=1676325583640), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 21:59:45,816 DEBUG [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c 2023-02-13 21:59:45,875 INFO [Listener at localhost.localdomain/34171] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/cluster_0697ba01-9152-a749-74f7-a697351678b4/zookeeper_0, clientPort=57335, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/cluster_0697ba01-9152-a749-74f7-a697351678b4/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/cluster_0697ba01-9152-a749-74f7-a697351678b4/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-13 21:59:45,889 INFO [Listener at localhost.localdomain/34171] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=57335 2023-02-13 21:59:45,896 INFO [Listener at localhost.localdomain/34171] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:45,898 INFO [Listener at localhost.localdomain/34171] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:46,536 INFO [Listener at localhost.localdomain/34171] util.FSUtils(479): Created version file at hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f with version=8 2023-02-13 21:59:46,536 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/hbase-staging 2023-02-13 21:59:46,804 INFO [Listener at localhost.localdomain/34171] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-13 21:59:47,184 INFO [Listener at localhost.localdomain/34171] client.ConnectionUtils(127): master/jenkins-hbase20:0 server-side Connection retries=45 2023-02-13 21:59:47,209 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:47,210 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:47,210 INFO [Listener at localhost.localdomain/34171] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 21:59:47,210 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:47,210 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 21:59:47,331 INFO [Listener at localhost.localdomain/34171] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 21:59:47,399 DEBUG [Listener at localhost.localdomain/34171] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-13 21:59:47,480 INFO [Listener at localhost.localdomain/34171] ipc.NettyRpcServer(120): Bind to /148.251.75.209:43889 2023-02-13 21:59:47,492 INFO [Listener at localhost.localdomain/34171] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:47,494 INFO [Listener at localhost.localdomain/34171] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:47,512 INFO [Listener at localhost.localdomain/34171] zookeeper.RecoverableZooKeeper(93): Process identifier=master:43889 connecting to ZooKeeper ensemble=127.0.0.1:57335 2023-02-13 21:59:47,546 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:438890x0, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 21:59:47,547 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:43889-0x1015dda45730000 connected 2023-02-13 21:59:47,571 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(164): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 21:59:47,572 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(164): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 21:59:47,575 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(164): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 21:59:47,582 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43889 2023-02-13 21:59:47,583 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43889 2023-02-13 21:59:47,583 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43889 2023-02-13 21:59:47,584 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43889 2023-02-13 21:59:47,584 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43889 2023-02-13 21:59:47,589 INFO [Listener at localhost.localdomain/34171] master.HMaster(439): hbase.rootdir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f, hbase.cluster.distributed=false 2023-02-13 21:59:47,651 INFO [Listener at localhost.localdomain/34171] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-02-13 21:59:47,651 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:47,651 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:47,651 INFO [Listener at localhost.localdomain/34171] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 21:59:47,651 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:47,651 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 21:59:47,655 INFO [Listener at localhost.localdomain/34171] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 21:59:47,658 INFO [Listener at localhost.localdomain/34171] ipc.NettyRpcServer(120): Bind to /148.251.75.209:46497 2023-02-13 21:59:47,660 INFO [Listener at localhost.localdomain/34171] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 21:59:47,666 DEBUG [Listener at localhost.localdomain/34171] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 21:59:47,668 INFO [Listener at localhost.localdomain/34171] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:47,669 INFO [Listener at localhost.localdomain/34171] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:47,672 INFO [Listener at localhost.localdomain/34171] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46497 connecting to ZooKeeper ensemble=127.0.0.1:57335 2023-02-13 21:59:47,676 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:464970x0, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 21:59:47,677 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(164): regionserver:464970x0, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 21:59:47,677 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:46497-0x1015dda45730001 connected 2023-02-13 21:59:47,678 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(164): regionserver:46497-0x1015dda45730001, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 21:59:47,679 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(164): regionserver:46497-0x1015dda45730001, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 21:59:47,679 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46497 2023-02-13 21:59:47,680 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46497 2023-02-13 21:59:47,680 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46497 2023-02-13 21:59:47,680 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46497 2023-02-13 21:59:47,680 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46497 2023-02-13 21:59:47,691 INFO [Listener at localhost.localdomain/34171] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-02-13 21:59:47,691 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:47,691 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:47,692 INFO [Listener at localhost.localdomain/34171] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 21:59:47,692 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:47,692 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 21:59:47,692 INFO [Listener at localhost.localdomain/34171] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 21:59:47,694 INFO [Listener at localhost.localdomain/34171] ipc.NettyRpcServer(120): Bind to /148.251.75.209:34119 2023-02-13 21:59:47,694 INFO [Listener at localhost.localdomain/34171] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 21:59:47,695 DEBUG [Listener at localhost.localdomain/34171] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 21:59:47,696 INFO [Listener at localhost.localdomain/34171] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:47,698 INFO [Listener at localhost.localdomain/34171] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:47,701 INFO [Listener at localhost.localdomain/34171] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34119 connecting to ZooKeeper ensemble=127.0.0.1:57335 2023-02-13 21:59:47,705 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:341190x0, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 21:59:47,706 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(164): regionserver:341190x0, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 21:59:47,706 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:34119-0x1015dda45730002 connected 2023-02-13 21:59:47,707 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(164): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 21:59:47,707 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(164): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 21:59:47,708 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34119 2023-02-13 21:59:47,708 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34119 2023-02-13 21:59:47,711 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34119 2023-02-13 21:59:47,711 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34119 2023-02-13 21:59:47,711 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34119 2023-02-13 21:59:47,722 INFO [Listener at localhost.localdomain/34171] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-02-13 21:59:47,723 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:47,723 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:47,723 INFO [Listener at localhost.localdomain/34171] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 21:59:47,723 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:47,723 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 21:59:47,723 INFO [Listener at localhost.localdomain/34171] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 21:59:47,725 INFO [Listener at localhost.localdomain/34171] ipc.NettyRpcServer(120): Bind to /148.251.75.209:45487 2023-02-13 21:59:47,725 INFO [Listener at localhost.localdomain/34171] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 21:59:47,727 DEBUG [Listener at localhost.localdomain/34171] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 21:59:47,728 INFO [Listener at localhost.localdomain/34171] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:47,729 INFO [Listener at localhost.localdomain/34171] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:47,730 INFO [Listener at localhost.localdomain/34171] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45487 connecting to ZooKeeper ensemble=127.0.0.1:57335 2023-02-13 21:59:47,734 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:454870x0, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 21:59:47,735 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(164): regionserver:454870x0, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 21:59:47,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:45487-0x1015dda45730003 connected 2023-02-13 21:59:47,736 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(164): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 21:59:47,737 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(164): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 21:59:47,739 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45487 2023-02-13 21:59:47,740 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45487 2023-02-13 21:59:47,743 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45487 2023-02-13 21:59:47,744 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45487 2023-02-13 21:59:47,746 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45487 2023-02-13 21:59:47,749 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase20.apache.org,43889,1676325586653 2023-02-13 21:59:47,759 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 21:59:47,760 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase20.apache.org,43889,1676325586653 2023-02-13 21:59:47,782 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:47,782 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:47,782 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:47,782 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:46497-0x1015dda45730001, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:47,782 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:47,784 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 21:59:47,785 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 21:59:47,785 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase20.apache.org,43889,1676325586653 from backup master directory 2023-02-13 21:59:47,791 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase20.apache.org,43889,1676325586653 2023-02-13 21:59:47,791 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 21:59:47,792 WARN [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 21:59:47,792 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase20.apache.org,43889,1676325586653 2023-02-13 21:59:47,795 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-13 21:59:47,796 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-13 21:59:47,886 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/hbase.id with ID: 611d7ab4-6c3d-48dd-a873-3bff225d9a89 2023-02-13 21:59:47,937 INFO [master/jenkins-hbase20:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:47,952 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:48,001 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x6adefa11 to 127.0.0.1:57335 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 21:59:48,032 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@294136bb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 21:59:48,053 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 21:59:48,055 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 21:59:48,071 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-13 21:59:48,071 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-13 21:59:48,073 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 21:59:48,076 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 21:59:48,077 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 21:59:48,107 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store-tmp 2023-02-13 21:59:48,172 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:48,173 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 21:59:48,173 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:48,174 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:48,174 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 21:59:48,174 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:48,174 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:48,174 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676325588173Disabling compacts and flushes for region at 1676325588173Disabling writes for close at 1676325588174 (+1 ms)Writing region close event to WAL at 1676325588174Closed at 1676325588174 2023-02-13 21:59:48,176 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/WALs/jenkins-hbase20.apache.org,43889,1676325586653 2023-02-13 21:59:48,198 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C43889%2C1676325586653, suffix=, logDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/WALs/jenkins-hbase20.apache.org,43889,1676325586653, archiveDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/oldWALs, maxLogs=10 2023-02-13 21:59:48,249 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK] 2023-02-13 21:59:48,249 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK] 2023-02-13 21:59:48,249 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK] 2023-02-13 21:59:48,256 DEBUG [RS-EventLoopGroup-5-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 21:59:48,318 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/WALs/jenkins-hbase20.apache.org,43889,1676325586653/jenkins-hbase20.apache.org%2C43889%2C1676325586653.1676325588207 2023-02-13 21:59:48,319 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK], DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK], DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK]] 2023-02-13 21:59:48,320 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 21:59:48,320 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:48,324 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:48,326 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:48,385 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:48,393 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 21:59:48,414 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 21:59:48,425 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:48,431 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:48,433 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:48,448 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:48,452 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 21:59:48,454 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63951875, jitterRate=-0.04704280197620392}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 21:59:48,454 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676325588353Initializing all the Stores at 1676325588355 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325588356 (+1 ms)Cleaning up temporary data from old regions at 1676325588439 (+83 ms)Cleaning up detritus from prior splits at 1676325588440 (+1 ms)Region opened successfully at 1676325588454 (+14 ms) 2023-02-13 21:59:48,456 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 21:59:48,474 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-13 21:59:48,474 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-13 21:59:48,476 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 21:59:48,478 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 21:59:48,509 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 30 msec 2023-02-13 21:59:48,509 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 21:59:48,531 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-13 21:59:48,536 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-13 21:59:48,564 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 21:59:48,569 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 21:59:48,570 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-13 21:59:48,576 INFO [master/jenkins-hbase20:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 21:59:48,581 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 21:59:48,584 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:48,586 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 21:59:48,586 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 21:59:48,599 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 21:59:48,603 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 21:59:48,603 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 21:59:48,603 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 21:59:48,603 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:46497-0x1015dda45730001, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 21:59:48,604 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:48,604 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase20.apache.org,43889,1676325586653, sessionid=0x1015dda45730000, setting cluster-up flag (Was=false) 2023-02-13 21:59:48,619 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:48,623 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 21:59:48,624 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,43889,1676325586653 2023-02-13 21:59:48,628 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:48,632 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 21:59:48,633 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,43889,1676325586653 2023-02-13 21:59:48,636 WARN [master/jenkins-hbase20:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.hbase-snapshot/.tmp 2023-02-13 21:59:48,650 INFO [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(952): ClusterId : 611d7ab4-6c3d-48dd-a873-3bff225d9a89 2023-02-13 21:59:48,650 INFO [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(952): ClusterId : 611d7ab4-6c3d-48dd-a873-3bff225d9a89 2023-02-13 21:59:48,650 INFO [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(952): ClusterId : 611d7ab4-6c3d-48dd-a873-3bff225d9a89 2023-02-13 21:59:48,657 DEBUG [RS:0;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 21:59:48,657 DEBUG [RS:1;jenkins-hbase20:34119] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 21:59:48,657 DEBUG [RS:2;jenkins-hbase20:45487] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 21:59:48,662 DEBUG [RS:1;jenkins-hbase20:34119] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 21:59:48,662 DEBUG [RS:0;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 21:59:48,662 DEBUG [RS:2;jenkins-hbase20:45487] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 21:59:48,663 DEBUG [RS:0;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 21:59:48,663 DEBUG [RS:1;jenkins-hbase20:34119] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 21:59:48,663 DEBUG [RS:2;jenkins-hbase20:45487] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 21:59:48,666 DEBUG [RS:0;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 21:59:48,666 DEBUG [RS:1;jenkins-hbase20:34119] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 21:59:48,666 DEBUG [RS:2;jenkins-hbase20:45487] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 21:59:48,673 DEBUG [RS:0;jenkins-hbase20:46497] zookeeper.ReadOnlyZKClient(139): Connect 0x7c316a08 to 127.0.0.1:57335 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 21:59:48,673 DEBUG [RS:2;jenkins-hbase20:45487] zookeeper.ReadOnlyZKClient(139): Connect 0x20e16025 to 127.0.0.1:57335 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 21:59:48,673 DEBUG [RS:1;jenkins-hbase20:34119] zookeeper.ReadOnlyZKClient(139): Connect 0x28ea3142 to 127.0.0.1:57335 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 21:59:48,684 DEBUG [RS:2;jenkins-hbase20:45487] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2e268a49, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 21:59:48,685 DEBUG [RS:0;jenkins-hbase20:46497] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@72f69bfe, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 21:59:48,685 DEBUG [RS:2;jenkins-hbase20:45487] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2a190b55, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 21:59:48,685 DEBUG [RS:0;jenkins-hbase20:46497] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@76c90178, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 21:59:48,685 DEBUG [RS:1;jenkins-hbase20:34119] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@262dca2f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 21:59:48,686 DEBUG [RS:1;jenkins-hbase20:34119] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@22125a32, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 21:59:48,707 DEBUG [RS:0;jenkins-hbase20:46497] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase20:46497 2023-02-13 21:59:48,709 DEBUG [RS:1;jenkins-hbase20:34119] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase20:34119 2023-02-13 21:59:48,711 INFO [RS:0;jenkins-hbase20:46497] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 21:59:48,711 INFO [RS:1;jenkins-hbase20:34119] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 21:59:48,712 INFO [RS:0;jenkins-hbase20:46497] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 21:59:48,712 INFO [RS:1;jenkins-hbase20:34119] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 21:59:48,712 DEBUG [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 21:59:48,712 DEBUG [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 21:59:48,713 DEBUG [RS:2;jenkins-hbase20:45487] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase20:45487 2023-02-13 21:59:48,713 INFO [RS:2;jenkins-hbase20:45487] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 21:59:48,713 INFO [RS:2;jenkins-hbase20:45487] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 21:59:48,713 DEBUG [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 21:59:48,715 INFO [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,43889,1676325586653 with isa=jenkins-hbase20.apache.org/148.251.75.209:45487, startcode=1676325587722 2023-02-13 21:59:48,715 INFO [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,43889,1676325586653 with isa=jenkins-hbase20.apache.org/148.251.75.209:34119, startcode=1676325587690 2023-02-13 21:59:48,715 INFO [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,43889,1676325586653 with isa=jenkins-hbase20.apache.org/148.251.75.209:46497, startcode=1676325587650 2023-02-13 21:59:48,737 DEBUG [RS:2;jenkins-hbase20:45487] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 21:59:48,737 DEBUG [RS:1;jenkins-hbase20:34119] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 21:59:48,737 DEBUG [RS:0;jenkins-hbase20:46497] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 21:59:48,792 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-13 21:59:48,802 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 21:59:48,802 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 21:59:48,802 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 21:59:48,802 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 21:59:48,802 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase20:0, corePoolSize=10, maxPoolSize=10 2023-02-13 21:59:48,802 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:48,803 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-13 21:59:48,803 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:48,803 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:34407, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 21:59:48,803 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:44119, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 21:59:48,803 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:38099, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 21:59:48,807 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325618807 2023-02-13 21:59:48,811 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 21:59:48,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 21:59:48,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43889] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 21:59:48,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43889] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 21:59:48,816 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 21:59:48,816 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-13 21:59:48,821 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 21:59:48,823 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 21:59:48,833 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 21:59:48,834 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 21:59:48,834 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 21:59:48,834 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 21:59:48,835 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:48,837 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 21:59:48,838 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 21:59:48,838 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 21:59:48,841 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 21:59:48,841 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 21:59:48,843 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676325588843,5,FailOnTimeoutGroup] 2023-02-13 21:59:48,844 DEBUG [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 21:59:48,844 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676325588844,5,FailOnTimeoutGroup] 2023-02-13 21:59:48,844 DEBUG [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 21:59:48,844 DEBUG [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 21:59:48,844 WARN [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-13 21:59:48,844 WARN [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-13 21:59:48,844 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:48,844 WARN [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-13 21:59:48,845 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 21:59:48,846 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:48,846 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:48,869 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 21:59:48,870 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 21:59:48,871 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f 2023-02-13 21:59:48,900 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:48,903 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 21:59:48,906 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/info 2023-02-13 21:59:48,907 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 21:59:48,907 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:48,908 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 21:59:48,910 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/rep_barrier 2023-02-13 21:59:48,911 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 21:59:48,912 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:48,912 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 21:59:48,914 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/table 2023-02-13 21:59:48,914 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 21:59:48,915 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:48,917 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740 2023-02-13 21:59:48,918 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740 2023-02-13 21:59:48,922 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 21:59:48,924 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 21:59:48,928 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 21:59:48,929 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60090724, jitterRate=-0.10457843542098999}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 21:59:48,929 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676325588901Initializing all the Stores at 1676325588902 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325588902Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325588902Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325588903 (+1 ms)Cleaning up temporary data from old regions at 1676325588920 (+17 ms)Cleaning up detritus from prior splits at 1676325588921 (+1 ms)Region opened successfully at 1676325588929 (+8 ms) 2023-02-13 21:59:48,929 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 21:59:48,929 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 21:59:48,929 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 21:59:48,929 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 21:59:48,929 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 21:59:48,932 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 21:59:48,933 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676325588929Disabling compacts and flushes for region at 1676325588929Disabling writes for close at 1676325588929Writing region close event to WAL at 1676325588932 (+3 ms)Closed at 1676325588932 2023-02-13 21:59:48,939 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 21:59:48,939 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-13 21:59:48,946 INFO [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,43889,1676325586653 with isa=jenkins-hbase20.apache.org/148.251.75.209:45487, startcode=1676325587722 2023-02-13 21:59:48,946 INFO [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,43889,1676325586653 with isa=jenkins-hbase20.apache.org/148.251.75.209:46497, startcode=1676325587650 2023-02-13 21:59:48,946 INFO [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,43889,1676325586653 with isa=jenkins-hbase20.apache.org/148.251.75.209:34119, startcode=1676325587690 2023-02-13 21:59:48,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 21:59:48,953 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43889] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:48,954 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43889] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:48,955 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:48,963 DEBUG [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f 2023-02-13 21:59:48,963 DEBUG [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f 2023-02-13 21:59:48,963 DEBUG [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f 2023-02-13 21:59:48,963 DEBUG [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:36521 2023-02-13 21:59:48,963 DEBUG [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:36521 2023-02-13 21:59:48,963 DEBUG [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 21:59:48,963 DEBUG [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:36521 2023-02-13 21:59:48,963 DEBUG [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 21:59:48,964 DEBUG [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 21:59:48,968 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 21:59:48,970 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 21:59:48,971 DEBUG [RS:1;jenkins-hbase20:34119] zookeeper.ZKUtil(162): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:48,971 DEBUG [RS:0;jenkins-hbase20:46497] zookeeper.ZKUtil(162): regionserver:46497-0x1015dda45730001, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:48,972 DEBUG [RS:2;jenkins-hbase20:45487] zookeeper.ZKUtil(162): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:48,971 WARN [RS:1;jenkins-hbase20:34119] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 21:59:48,972 INFO [RS:1;jenkins-hbase20:34119] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 21:59:48,972 DEBUG [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:48,972 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 21:59:48,972 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,46497,1676325587650] 2023-02-13 21:59:48,973 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,45487,1676325587722] 2023-02-13 21:59:48,973 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,34119,1676325587690] 2023-02-13 21:59:48,972 WARN [RS:0;jenkins-hbase20:46497] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 21:59:48,972 WARN [RS:2;jenkins-hbase20:45487] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 21:59:48,973 INFO [RS:0;jenkins-hbase20:46497] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 21:59:48,974 INFO [RS:2;jenkins-hbase20:45487] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 21:59:48,974 DEBUG [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:48,974 DEBUG [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:48,986 DEBUG [RS:1;jenkins-hbase20:34119] zookeeper.ZKUtil(162): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:48,986 DEBUG [RS:0;jenkins-hbase20:46497] zookeeper.ZKUtil(162): regionserver:46497-0x1015dda45730001, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:48,986 DEBUG [RS:2;jenkins-hbase20:45487] zookeeper.ZKUtil(162): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:48,987 DEBUG [RS:1;jenkins-hbase20:34119] zookeeper.ZKUtil(162): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:48,987 DEBUG [RS:0;jenkins-hbase20:46497] zookeeper.ZKUtil(162): regionserver:46497-0x1015dda45730001, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:48,987 DEBUG [RS:2;jenkins-hbase20:45487] zookeeper.ZKUtil(162): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:48,987 DEBUG [RS:1;jenkins-hbase20:34119] zookeeper.ZKUtil(162): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:48,987 DEBUG [RS:0;jenkins-hbase20:46497] zookeeper.ZKUtil(162): regionserver:46497-0x1015dda45730001, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:48,987 DEBUG [RS:2;jenkins-hbase20:45487] zookeeper.ZKUtil(162): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:48,997 DEBUG [RS:2;jenkins-hbase20:45487] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 21:59:48,997 DEBUG [RS:1;jenkins-hbase20:34119] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 21:59:48,997 DEBUG [RS:0;jenkins-hbase20:46497] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 21:59:49,007 INFO [RS:1;jenkins-hbase20:34119] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 21:59:49,007 INFO [RS:0;jenkins-hbase20:46497] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 21:59:49,007 INFO [RS:2;jenkins-hbase20:45487] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 21:59:49,031 INFO [RS:1;jenkins-hbase20:34119] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 21:59:49,031 INFO [RS:0;jenkins-hbase20:46497] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 21:59:49,031 INFO [RS:2;jenkins-hbase20:45487] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 21:59:49,035 INFO [RS:1;jenkins-hbase20:34119] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 21:59:49,035 INFO [RS:0;jenkins-hbase20:46497] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 21:59:49,035 INFO [RS:2;jenkins-hbase20:45487] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 21:59:49,035 INFO [RS:0;jenkins-hbase20:46497] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,035 INFO [RS:1;jenkins-hbase20:34119] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,036 INFO [RS:2;jenkins-hbase20:45487] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,038 INFO [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 21:59:49,038 INFO [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 21:59:49,038 INFO [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 21:59:49,048 INFO [RS:2;jenkins-hbase20:45487] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,048 INFO [RS:1;jenkins-hbase20:34119] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,048 INFO [RS:0;jenkins-hbase20:46497] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,048 DEBUG [RS:1;jenkins-hbase20:34119] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,048 DEBUG [RS:2;jenkins-hbase20:45487] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,049 DEBUG [RS:1;jenkins-hbase20:34119] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,049 DEBUG [RS:2;jenkins-hbase20:45487] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,049 DEBUG [RS:0;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,049 DEBUG [RS:2;jenkins-hbase20:45487] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,049 DEBUG [RS:1;jenkins-hbase20:34119] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,049 DEBUG [RS:2;jenkins-hbase20:45487] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,049 DEBUG [RS:1;jenkins-hbase20:34119] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,050 DEBUG [RS:2;jenkins-hbase20:45487] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,050 DEBUG [RS:1;jenkins-hbase20:34119] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,050 DEBUG [RS:2;jenkins-hbase20:45487] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-13 21:59:49,050 DEBUG [RS:1;jenkins-hbase20:34119] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-13 21:59:49,050 DEBUG [RS:2;jenkins-hbase20:45487] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,050 DEBUG [RS:1;jenkins-hbase20:34119] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,049 DEBUG [RS:0;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,050 DEBUG [RS:1;jenkins-hbase20:34119] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,050 DEBUG [RS:0;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,050 DEBUG [RS:2;jenkins-hbase20:45487] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,050 DEBUG [RS:0;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,050 DEBUG [RS:1;jenkins-hbase20:34119] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,050 DEBUG [RS:0;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,050 DEBUG [RS:2;jenkins-hbase20:45487] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,050 DEBUG [RS:0;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-13 21:59:49,050 DEBUG [RS:1;jenkins-hbase20:34119] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,051 DEBUG [RS:0;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,051 DEBUG [RS:2;jenkins-hbase20:45487] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,051 DEBUG [RS:0;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,054 DEBUG [RS:0;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,054 DEBUG [RS:0;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:49,054 INFO [RS:1;jenkins-hbase20:34119] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,054 INFO [RS:2;jenkins-hbase20:45487] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,055 INFO [RS:1;jenkins-hbase20:34119] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,055 INFO [RS:2;jenkins-hbase20:45487] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,055 INFO [RS:1;jenkins-hbase20:34119] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,055 INFO [RS:2;jenkins-hbase20:45487] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,059 INFO [RS:0;jenkins-hbase20:46497] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,059 INFO [RS:0;jenkins-hbase20:46497] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,059 INFO [RS:0;jenkins-hbase20:46497] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,072 INFO [RS:0;jenkins-hbase20:46497] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 21:59:49,074 INFO [RS:2;jenkins-hbase20:45487] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 21:59:49,074 INFO [RS:1;jenkins-hbase20:34119] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 21:59:49,075 INFO [RS:1;jenkins-hbase20:34119] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,34119,1676325587690-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,075 INFO [RS:0;jenkins-hbase20:46497] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,46497,1676325587650-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,075 INFO [RS:2;jenkins-hbase20:45487] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,45487,1676325587722-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,097 INFO [RS:0;jenkins-hbase20:46497] regionserver.Replication(203): jenkins-hbase20.apache.org,46497,1676325587650 started 2023-02-13 21:59:49,097 INFO [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,46497,1676325587650, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:46497, sessionid=0x1015dda45730001 2023-02-13 21:59:49,099 INFO [RS:1;jenkins-hbase20:34119] regionserver.Replication(203): jenkins-hbase20.apache.org,34119,1676325587690 started 2023-02-13 21:59:49,099 INFO [RS:2;jenkins-hbase20:45487] regionserver.Replication(203): jenkins-hbase20.apache.org,45487,1676325587722 started 2023-02-13 21:59:49,099 INFO [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,34119,1676325587690, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:34119, sessionid=0x1015dda45730002 2023-02-13 21:59:49,099 DEBUG [RS:0;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 21:59:49,099 DEBUG [RS:1;jenkins-hbase20:34119] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 21:59:49,099 INFO [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,45487,1676325587722, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:45487, sessionid=0x1015dda45730003 2023-02-13 21:59:49,099 DEBUG [RS:1;jenkins-hbase20:34119] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:49,100 DEBUG [RS:2;jenkins-hbase20:45487] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 21:59:49,100 DEBUG [RS:1;jenkins-hbase20:34119] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,34119,1676325587690' 2023-02-13 21:59:49,099 DEBUG [RS:0;jenkins-hbase20:46497] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:49,100 DEBUG [RS:1;jenkins-hbase20:34119] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 21:59:49,100 DEBUG [RS:0;jenkins-hbase20:46497] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,46497,1676325587650' 2023-02-13 21:59:49,100 DEBUG [RS:2;jenkins-hbase20:45487] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:49,101 DEBUG [RS:2;jenkins-hbase20:45487] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,45487,1676325587722' 2023-02-13 21:59:49,101 DEBUG [RS:2;jenkins-hbase20:45487] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 21:59:49,100 DEBUG [RS:0;jenkins-hbase20:46497] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 21:59:49,101 DEBUG [RS:2;jenkins-hbase20:45487] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 21:59:49,101 DEBUG [RS:0;jenkins-hbase20:46497] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 21:59:49,101 DEBUG [RS:1;jenkins-hbase20:34119] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 21:59:49,102 DEBUG [RS:0;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 21:59:49,102 DEBUG [RS:1;jenkins-hbase20:34119] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 21:59:49,102 DEBUG [RS:0;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 21:59:49,102 DEBUG [RS:1;jenkins-hbase20:34119] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 21:59:49,102 DEBUG [RS:0;jenkins-hbase20:46497] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:49,102 DEBUG [RS:1;jenkins-hbase20:34119] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:49,103 DEBUG [RS:0;jenkins-hbase20:46497] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,46497,1676325587650' 2023-02-13 21:59:49,103 DEBUG [RS:0;jenkins-hbase20:46497] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 21:59:49,103 DEBUG [RS:1;jenkins-hbase20:34119] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,34119,1676325587690' 2023-02-13 21:59:49,103 DEBUG [RS:2;jenkins-hbase20:45487] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 21:59:49,103 DEBUG [RS:1;jenkins-hbase20:34119] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 21:59:49,103 DEBUG [RS:2;jenkins-hbase20:45487] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 21:59:49,103 DEBUG [RS:2;jenkins-hbase20:45487] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:49,103 DEBUG [RS:2;jenkins-hbase20:45487] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,45487,1676325587722' 2023-02-13 21:59:49,103 DEBUG [RS:2;jenkins-hbase20:45487] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 21:59:49,103 DEBUG [RS:0;jenkins-hbase20:46497] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 21:59:49,103 DEBUG [RS:1;jenkins-hbase20:34119] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 21:59:49,104 DEBUG [RS:2;jenkins-hbase20:45487] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 21:59:49,104 DEBUG [RS:0;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 21:59:49,104 INFO [RS:0;jenkins-hbase20:46497] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 21:59:49,104 DEBUG [RS:1;jenkins-hbase20:34119] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 21:59:49,104 INFO [RS:0;jenkins-hbase20:46497] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 21:59:49,104 DEBUG [RS:2;jenkins-hbase20:45487] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 21:59:49,104 INFO [RS:1;jenkins-hbase20:34119] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 21:59:49,105 INFO [RS:2;jenkins-hbase20:45487] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 21:59:49,105 INFO [RS:1;jenkins-hbase20:34119] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 21:59:49,106 INFO [RS:2;jenkins-hbase20:45487] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 21:59:49,125 DEBUG [jenkins-hbase20:43889] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-13 21:59:49,128 DEBUG [jenkins-hbase20:43889] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 21:59:49,134 DEBUG [jenkins-hbase20:43889] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 21:59:49,134 DEBUG [jenkins-hbase20:43889] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 21:59:49,134 DEBUG [jenkins-hbase20:43889] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 21:59:49,134 DEBUG [jenkins-hbase20:43889] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 21:59:49,137 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,46497,1676325587650, state=OPENING 2023-02-13 21:59:49,146 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-13 21:59:49,147 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:49,148 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 21:59:49,152 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,46497,1676325587650}] 2023-02-13 21:59:49,214 INFO [RS:1;jenkins-hbase20:34119] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C34119%2C1676325587690, suffix=, logDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,34119,1676325587690, archiveDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/oldWALs, maxLogs=32 2023-02-13 21:59:49,214 INFO [RS:2;jenkins-hbase20:45487] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C45487%2C1676325587722, suffix=, logDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,45487,1676325587722, archiveDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/oldWALs, maxLogs=32 2023-02-13 21:59:49,214 INFO [RS:0;jenkins-hbase20:46497] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C46497%2C1676325587650, suffix=, logDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,46497,1676325587650, archiveDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/oldWALs, maxLogs=32 2023-02-13 21:59:49,237 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK] 2023-02-13 21:59:49,237 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK] 2023-02-13 21:59:49,238 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK] 2023-02-13 21:59:49,239 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK] 2023-02-13 21:59:49,240 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK] 2023-02-13 21:59:49,240 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK] 2023-02-13 21:59:49,247 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK] 2023-02-13 21:59:49,248 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK] 2023-02-13 21:59:49,247 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK] 2023-02-13 21:59:49,262 INFO [RS:2;jenkins-hbase20:45487] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,45487,1676325587722/jenkins-hbase20.apache.org%2C45487%2C1676325587722.1676325589217 2023-02-13 21:59:49,262 INFO [RS:1;jenkins-hbase20:34119] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,34119,1676325587690/jenkins-hbase20.apache.org%2C34119%2C1676325587690.1676325589217 2023-02-13 21:59:49,263 INFO [RS:0;jenkins-hbase20:46497] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,46497,1676325587650/jenkins-hbase20.apache.org%2C46497%2C1676325587650.1676325589217 2023-02-13 21:59:49,263 DEBUG [RS:2;jenkins-hbase20:45487] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK], DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK], DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK]] 2023-02-13 21:59:49,263 DEBUG [RS:1;jenkins-hbase20:34119] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK], DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK], DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK]] 2023-02-13 21:59:49,263 DEBUG [RS:0;jenkins-hbase20:46497] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK], DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK], DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK]] 2023-02-13 21:59:49,335 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:49,337 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 21:59:49,340 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:33522, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 21:59:49,356 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 21:59:49,357 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 21:59:49,361 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C46497%2C1676325587650.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,46497,1676325587650, archiveDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/oldWALs, maxLogs=32 2023-02-13 21:59:49,388 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK] 2023-02-13 21:59:49,389 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK] 2023-02-13 21:59:49,389 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK] 2023-02-13 21:59:49,399 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,46497,1676325587650/jenkins-hbase20.apache.org%2C46497%2C1676325587650.meta.1676325589363.meta 2023-02-13 21:59:49,400 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK], DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK], DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK]] 2023-02-13 21:59:49,400 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 21:59:49,401 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 21:59:49,416 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 21:59:49,420 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 21:59:49,425 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 21:59:49,425 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:49,425 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 21:59:49,425 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 21:59:49,428 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 21:59:49,429 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/info 2023-02-13 21:59:49,430 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/info 2023-02-13 21:59:49,430 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 21:59:49,431 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:49,431 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 21:59:49,433 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/rep_barrier 2023-02-13 21:59:49,433 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/rep_barrier 2023-02-13 21:59:49,434 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 21:59:49,435 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:49,435 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 21:59:49,437 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/table 2023-02-13 21:59:49,437 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/table 2023-02-13 21:59:49,437 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 21:59:49,438 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:49,441 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740 2023-02-13 21:59:49,444 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740 2023-02-13 21:59:49,448 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 21:59:49,450 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 21:59:49,451 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62513325, jitterRate=-0.06847886741161346}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 21:59:49,452 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676325589425Writing region info on filesystem at 1676325589425Initializing all the Stores at 1676325589427 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325589427Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325589427Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325589427Cleaning up temporary data from old regions at 1676325589446 (+19 ms)Cleaning up detritus from prior splits at 1676325589446Running coprocessor post-open hooks at 1676325589451 (+5 ms)Region opened successfully at 1676325589451 2023-02-13 21:59:49,460 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676325589328 2023-02-13 21:59:49,477 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 21:59:49,478 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 21:59:49,479 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,46497,1676325587650, state=OPEN 2023-02-13 21:59:49,482 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 21:59:49,482 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 21:59:49,487 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-13 21:59:49,487 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,46497,1676325587650 in 330 msec 2023-02-13 21:59:49,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-13 21:59:49,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 537 msec 2023-02-13 21:59:49,503 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 799 msec 2023-02-13 21:59:49,503 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676325587778, completionTime=-1 2023-02-13 21:59:49,503 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-13 21:59:49,503 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 21:59:49,559 DEBUG [hconnection-0x3579db39-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 21:59:49,563 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:33528, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 21:59:49,578 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 21:59:49,578 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325649578 2023-02-13 21:59:49,578 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325709578 2023-02-13 21:59:49,578 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 74 msec 2023-02-13 21:59:49,605 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,43889,1676325586653-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,606 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,43889,1676325586653-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,606 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,43889,1676325586653-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,607 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase20:43889, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,608 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:49,615 DEBUG [master/jenkins-hbase20:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-13 21:59:49,623 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-13 21:59:49,625 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 21:59:49,634 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-13 21:59:49,637 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 21:59:49,640 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 21:59:49,661 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/hbase/namespace/b2169e2475351a310eb3323519b34622 2023-02-13 21:59:49,663 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/hbase/namespace/b2169e2475351a310eb3323519b34622 empty. 2023-02-13 21:59:49,664 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/hbase/namespace/b2169e2475351a310eb3323519b34622 2023-02-13 21:59:49,664 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-13 21:59:49,701 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-13 21:59:49,705 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => b2169e2475351a310eb3323519b34622, NAME => 'hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp 2023-02-13 21:59:49,727 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:49,727 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing b2169e2475351a310eb3323519b34622, disabling compactions & flushes 2023-02-13 21:59:49,727 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622. 2023-02-13 21:59:49,727 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622. 2023-02-13 21:59:49,728 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622. after waiting 0 ms 2023-02-13 21:59:49,728 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622. 2023-02-13 21:59:49,728 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622. 2023-02-13 21:59:49,728 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for b2169e2475351a310eb3323519b34622: Waiting for close lock at 1676325589727Disabling compacts and flushes for region at 1676325589727Disabling writes for close at 1676325589728 (+1 ms)Writing region close event to WAL at 1676325589728Closed at 1676325589728 2023-02-13 21:59:49,732 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 21:59:49,746 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676325589735"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325589735"}]},"ts":"1676325589735"} 2023-02-13 21:59:49,770 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 21:59:49,773 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 21:59:49,776 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325589773"}]},"ts":"1676325589773"} 2023-02-13 21:59:49,782 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-13 21:59:49,785 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 21:59:49,786 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 21:59:49,786 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 21:59:49,786 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 21:59:49,786 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 21:59:49,789 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=b2169e2475351a310eb3323519b34622, ASSIGN}] 2023-02-13 21:59:49,792 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=b2169e2475351a310eb3323519b34622, ASSIGN 2023-02-13 21:59:49,793 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=b2169e2475351a310eb3323519b34622, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,34119,1676325587690; forceNewPlan=false, retain=false 2023-02-13 21:59:49,945 INFO [jenkins-hbase20:43889] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 21:59:49,946 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=b2169e2475351a310eb3323519b34622, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:49,947 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676325589946"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325589946"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325589946"}]},"ts":"1676325589946"} 2023-02-13 21:59:49,951 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure b2169e2475351a310eb3323519b34622, server=jenkins-hbase20.apache.org,34119,1676325587690}] 2023-02-13 21:59:50,107 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:50,108 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 21:59:50,112 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:35010, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 21:59:50,122 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622. 2023-02-13 21:59:50,122 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b2169e2475351a310eb3323519b34622, NAME => 'hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622.', STARTKEY => '', ENDKEY => ''} 2023-02-13 21:59:50,123 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace b2169e2475351a310eb3323519b34622 2023-02-13 21:59:50,123 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:50,123 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for b2169e2475351a310eb3323519b34622 2023-02-13 21:59:50,123 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for b2169e2475351a310eb3323519b34622 2023-02-13 21:59:50,126 INFO [StoreOpener-b2169e2475351a310eb3323519b34622-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region b2169e2475351a310eb3323519b34622 2023-02-13 21:59:50,128 DEBUG [StoreOpener-b2169e2475351a310eb3323519b34622-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/namespace/b2169e2475351a310eb3323519b34622/info 2023-02-13 21:59:50,128 DEBUG [StoreOpener-b2169e2475351a310eb3323519b34622-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/namespace/b2169e2475351a310eb3323519b34622/info 2023-02-13 21:59:50,128 INFO [StoreOpener-b2169e2475351a310eb3323519b34622-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b2169e2475351a310eb3323519b34622 columnFamilyName info 2023-02-13 21:59:50,129 INFO [StoreOpener-b2169e2475351a310eb3323519b34622-1] regionserver.HStore(310): Store=b2169e2475351a310eb3323519b34622/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:50,131 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/namespace/b2169e2475351a310eb3323519b34622 2023-02-13 21:59:50,133 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/namespace/b2169e2475351a310eb3323519b34622 2023-02-13 21:59:50,137 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for b2169e2475351a310eb3323519b34622 2023-02-13 21:59:50,141 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/namespace/b2169e2475351a310eb3323519b34622/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 21:59:50,142 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened b2169e2475351a310eb3323519b34622; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71898561, jitterRate=0.07137204706668854}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 21:59:50,142 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for b2169e2475351a310eb3323519b34622: Running coprocessor pre-open hook at 1676325590124Writing region info on filesystem at 1676325590124Initializing all the Stores at 1676325590125 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325590125Cleaning up temporary data from old regions at 1676325590134 (+9 ms)Cleaning up detritus from prior splits at 1676325590134Running coprocessor post-open hooks at 1676325590142 (+8 ms)Region opened successfully at 1676325590142 2023-02-13 21:59:50,144 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622., pid=6, masterSystemTime=1676325590107 2023-02-13 21:59:50,150 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622. 2023-02-13 21:59:50,151 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622. 2023-02-13 21:59:50,152 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=b2169e2475351a310eb3323519b34622, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:50,153 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676325590152"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325590152"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325590152"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325590152"}]},"ts":"1676325590152"} 2023-02-13 21:59:50,162 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-13 21:59:50,162 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure b2169e2475351a310eb3323519b34622, server=jenkins-hbase20.apache.org,34119,1676325587690 in 207 msec 2023-02-13 21:59:50,165 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-13 21:59:50,166 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=b2169e2475351a310eb3323519b34622, ASSIGN in 374 msec 2023-02-13 21:59:50,167 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 21:59:50,168 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325590167"}]},"ts":"1676325590167"} 2023-02-13 21:59:50,170 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-13 21:59:50,173 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 21:59:50,176 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 547 msec 2023-02-13 21:59:50,237 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-13 21:59:50,238 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-13 21:59:50,239 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:50,259 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 21:59:50,263 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:35022, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 21:59:50,279 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-13 21:59:50,299 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 21:59:50,306 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 35 msec 2023-02-13 21:59:50,314 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-13 21:59:50,325 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 21:59:50,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-02-13 21:59:50,342 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 21:59:50,343 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 21:59:50,343 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.551sec 2023-02-13 21:59:50,346 INFO [master/jenkins-hbase20:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 21:59:50,347 INFO [master/jenkins-hbase20:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 21:59:50,347 INFO [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 21:59:50,348 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,43889,1676325586653-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 21:59:50,350 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,43889,1676325586653-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 21:59:50,357 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ReadOnlyZKClient(139): Connect 0x6263f8ce to 127.0.0.1:57335 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 21:59:50,359 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 21:59:50,362 DEBUG [Listener at localhost.localdomain/34171] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2ccab203, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 21:59:50,376 DEBUG [hconnection-0x2653f680-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 21:59:50,387 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:33544, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 21:59:50,396 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase20.apache.org,43889,1676325586653 2023-02-13 21:59:50,400 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-13 21:59:50,401 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:50,402 INFO [Listener at localhost.localdomain/34171] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-02-13 21:59:50,402 INFO [Listener at localhost.localdomain/34171] master.TestRetainAssignmentOnRestart(212): Creating tables 2023-02-13 21:59:50,411 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 21:59:50,414 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:39002, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 21:59:50,426 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.HMaster$4(2102): Client=jenkins//148.251.75.209 create 'restartTableOne', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 21:59:50,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=restartTableOne 2023-02-13 21:59:50,432 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=restartTableOne execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 21:59:50,434 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=restartTableOne execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 21:59:50,436 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.MasterRpcServices(697): Client=jenkins//148.251.75.209 procedure request for creating table: namespace: "default" qualifier: "restartTableOne" procId is: 9 2023-02-13 21:59:50,437 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/default/restartTableOne/364f7b221bf1134db4d8b7aca4f9c7a0 2023-02-13 21:59:50,438 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/default/restartTableOne/364f7b221bf1134db4d8b7aca4f9c7a0 empty. 2023-02-13 21:59:50,441 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/default/restartTableOne/364f7b221bf1134db4d8b7aca4f9c7a0 2023-02-13 21:59:50,441 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived restartTableOne regions 2023-02-13 21:59:50,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 21:59:50,464 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/default/restartTableOne/.tabledesc/.tableinfo.0000000001 2023-02-13 21:59:50,466 INFO [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(7671): creating {ENCODED => 364f7b221bf1134db4d8b7aca4f9c7a0, NAME => 'restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='restartTableOne', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp 2023-02-13 21:59:50,482 DEBUG [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(865): Instantiated restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:50,482 DEBUG [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(1603): Closing 364f7b221bf1134db4d8b7aca4f9c7a0, disabling compactions & flushes 2023-02-13 21:59:50,482 INFO [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(1625): Closing region restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0. 2023-02-13 21:59:50,483 DEBUG [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0. 2023-02-13 21:59:50,483 DEBUG [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(1713): Acquired close lock on restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0. after waiting 0 ms 2023-02-13 21:59:50,483 DEBUG [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(1723): Updates disabled for region restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0. 2023-02-13 21:59:50,483 INFO [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(1837): Closed restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0. 2023-02-13 21:59:50,483 DEBUG [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(1557): Region close journal for 364f7b221bf1134db4d8b7aca4f9c7a0: Waiting for close lock at 1676325590482Disabling compacts and flushes for region at 1676325590482Disabling writes for close at 1676325590483 (+1 ms)Writing region close event to WAL at 1676325590483Closed at 1676325590483 2023-02-13 21:59:50,487 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=restartTableOne execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 21:59:50,489 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325590489"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325590489"}]},"ts":"1676325590489"} 2023-02-13 21:59:50,491 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 21:59:50,493 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=restartTableOne execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 21:59:50,493 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"restartTableOne","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325590493"}]},"ts":"1676325590493"} 2023-02-13 21:59:50,496 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=restartTableOne, state=ENABLING in hbase:meta 2023-02-13 21:59:50,499 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 21:59:50,500 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 21:59:50,500 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 21:59:50,500 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 21:59:50,500 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 21:59:50,500 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableOne, region=364f7b221bf1134db4d8b7aca4f9c7a0, ASSIGN}] 2023-02-13 21:59:50,503 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableOne, region=364f7b221bf1134db4d8b7aca4f9c7a0, ASSIGN 2023-02-13 21:59:50,504 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=restartTableOne, region=364f7b221bf1134db4d8b7aca4f9c7a0, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,34119,1676325587690; forceNewPlan=false, retain=false 2023-02-13 21:59:50,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 21:59:50,655 INFO [jenkins-hbase20:43889] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 21:59:50,656 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=364f7b221bf1134db4d8b7aca4f9c7a0, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:50,656 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325590656"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325590656"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325590656"}]},"ts":"1676325590656"} 2023-02-13 21:59:50,659 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 364f7b221bf1134db4d8b7aca4f9c7a0, server=jenkins-hbase20.apache.org,34119,1676325587690}] 2023-02-13 21:59:50,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 21:59:50,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0. 2023-02-13 21:59:50,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 364f7b221bf1134db4d8b7aca4f9c7a0, NAME => 'restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0.', STARTKEY => '', ENDKEY => ''} 2023-02-13 21:59:50,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table restartTableOne 364f7b221bf1134db4d8b7aca4f9c7a0 2023-02-13 21:59:50,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:50,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 364f7b221bf1134db4d8b7aca4f9c7a0 2023-02-13 21:59:50,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 364f7b221bf1134db4d8b7aca4f9c7a0 2023-02-13 21:59:50,822 INFO [StoreOpener-364f7b221bf1134db4d8b7aca4f9c7a0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family of region 364f7b221bf1134db4d8b7aca4f9c7a0 2023-02-13 21:59:50,824 DEBUG [StoreOpener-364f7b221bf1134db4d8b7aca4f9c7a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableOne/364f7b221bf1134db4d8b7aca4f9c7a0/family 2023-02-13 21:59:50,824 DEBUG [StoreOpener-364f7b221bf1134db4d8b7aca4f9c7a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableOne/364f7b221bf1134db4d8b7aca4f9c7a0/family 2023-02-13 21:59:50,825 INFO [StoreOpener-364f7b221bf1134db4d8b7aca4f9c7a0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 364f7b221bf1134db4d8b7aca4f9c7a0 columnFamilyName family 2023-02-13 21:59:50,826 INFO [StoreOpener-364f7b221bf1134db4d8b7aca4f9c7a0-1] regionserver.HStore(310): Store=364f7b221bf1134db4d8b7aca4f9c7a0/family, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:50,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableOne/364f7b221bf1134db4d8b7aca4f9c7a0 2023-02-13 21:59:50,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableOne/364f7b221bf1134db4d8b7aca4f9c7a0 2023-02-13 21:59:50,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 364f7b221bf1134db4d8b7aca4f9c7a0 2023-02-13 21:59:50,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableOne/364f7b221bf1134db4d8b7aca4f9c7a0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 21:59:50,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 364f7b221bf1134db4d8b7aca4f9c7a0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67483507, jitterRate=0.0055826157331466675}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 21:59:50,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 364f7b221bf1134db4d8b7aca4f9c7a0: Running coprocessor pre-open hook at 1676325590820Writing region info on filesystem at 1676325590820Initializing all the Stores at 1676325590821 (+1 ms)Instantiating store for column family {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325590821Cleaning up temporary data from old regions at 1676325590831 (+10 ms)Cleaning up detritus from prior splits at 1676325590832 (+1 ms)Running coprocessor post-open hooks at 1676325590839 (+7 ms)Region opened successfully at 1676325590839 2023-02-13 21:59:50,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0., pid=11, masterSystemTime=1676325590812 2023-02-13 21:59:50,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0. 2023-02-13 21:59:50,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0. 2023-02-13 21:59:50,843 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=364f7b221bf1134db4d8b7aca4f9c7a0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:50,844 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325590843"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325590843"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325590843"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325590843"}]},"ts":"1676325590843"} 2023-02-13 21:59:50,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-13 21:59:50,852 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 364f7b221bf1134db4d8b7aca4f9c7a0, server=jenkins-hbase20.apache.org,34119,1676325587690 in 188 msec 2023-02-13 21:59:50,855 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-13 21:59:50,856 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=restartTableOne, region=364f7b221bf1134db4d8b7aca4f9c7a0, ASSIGN in 352 msec 2023-02-13 21:59:50,857 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=restartTableOne execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 21:59:50,858 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"restartTableOne","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325590858"}]},"ts":"1676325590858"} 2023-02-13 21:59:50,861 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=restartTableOne, state=ENABLED in hbase:meta 2023-02-13 21:59:50,864 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=restartTableOne execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 21:59:50,867 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=restartTableOne in 437 msec 2023-02-13 21:59:51,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 21:59:51,057 INFO [Listener at localhost.localdomain/34171] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:restartTableOne, procId: 9 completed 2023-02-13 21:59:51,057 DEBUG [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(3430): Waiting until all regions of table restartTableOne get assigned. Timeout = 60000ms 2023-02-13 21:59:51,059 INFO [Listener at localhost.localdomain/34171] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 21:59:51,064 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(3484): All regions for table restartTableOne assigned to meta. Checking AM states. 2023-02-13 21:59:51,064 INFO [Listener at localhost.localdomain/34171] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 21:59:51,064 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(3504): All regions for table restartTableOne assigned. 2023-02-13 21:59:51,067 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.HMaster$4(2102): Client=jenkins//148.251.75.209 create 'restartTableTwo', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 21:59:51,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=restartTableTwo 2023-02-13 21:59:51,071 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=restartTableTwo execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 21:59:51,071 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.MasterRpcServices(697): Client=jenkins//148.251.75.209 procedure request for creating table: namespace: "default" qualifier: "restartTableTwo" procId is: 12 2023-02-13 21:59:51,072 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=restartTableTwo execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 21:59:51,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.MasterRpcServices(1227): Checking to see if procedure is done pid=12 2023-02-13 21:59:51,075 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/default/restartTableTwo/6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:51,076 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/default/restartTableTwo/6bd825938f619ef4084c64b1f66ce2a6 empty. 2023-02-13 21:59:51,076 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/default/restartTableTwo/6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:51,076 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived restartTableTwo regions 2023-02-13 21:59:51,099 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/default/restartTableTwo/.tabledesc/.tableinfo.0000000001 2023-02-13 21:59:51,101 INFO [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(7671): creating {ENCODED => 6bd825938f619ef4084c64b1f66ce2a6, NAME => 'restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='restartTableTwo', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp 2023-02-13 21:59:51,120 DEBUG [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(865): Instantiated restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:51,120 DEBUG [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(1603): Closing 6bd825938f619ef4084c64b1f66ce2a6, disabling compactions & flushes 2023-02-13 21:59:51,120 INFO [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(1625): Closing region restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:51,120 DEBUG [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:51,120 DEBUG [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(1713): Acquired close lock on restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. after waiting 0 ms 2023-02-13 21:59:51,120 DEBUG [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(1723): Updates disabled for region restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:51,120 INFO [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(1837): Closed restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:51,120 DEBUG [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(1557): Region close journal for 6bd825938f619ef4084c64b1f66ce2a6: Waiting for close lock at 1676325591120Disabling compacts and flushes for region at 1676325591120Disabling writes for close at 1676325591120Writing region close event to WAL at 1676325591120Closed at 1676325591120 2023-02-13 21:59:51,125 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=restartTableTwo execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 21:59:51,128 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325591128"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325591128"}]},"ts":"1676325591128"} 2023-02-13 21:59:51,131 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 21:59:51,132 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=restartTableTwo execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 21:59:51,132 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"restartTableTwo","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325591132"}]},"ts":"1676325591132"} 2023-02-13 21:59:51,135 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=restartTableTwo, state=ENABLING in hbase:meta 2023-02-13 21:59:51,138 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 21:59:51,139 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 21:59:51,139 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 21:59:51,139 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 21:59:51,139 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 21:59:51,140 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableTwo, region=6bd825938f619ef4084c64b1f66ce2a6, ASSIGN}] 2023-02-13 21:59:51,142 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableTwo, region=6bd825938f619ef4084c64b1f66ce2a6, ASSIGN 2023-02-13 21:59:51,144 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=restartTableTwo, region=6bd825938f619ef4084c64b1f66ce2a6, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,46497,1676325587650; forceNewPlan=false, retain=false 2023-02-13 21:59:51,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.MasterRpcServices(1227): Checking to see if procedure is done pid=12 2023-02-13 21:59:51,294 INFO [jenkins-hbase20:43889] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 21:59:51,295 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=6bd825938f619ef4084c64b1f66ce2a6, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:51,295 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325591295"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325591295"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325591295"}]},"ts":"1676325591295"} 2023-02-13 21:59:51,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 6bd825938f619ef4084c64b1f66ce2a6, server=jenkins-hbase20.apache.org,46497,1676325587650}] 2023-02-13 21:59:51,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.MasterRpcServices(1227): Checking to see if procedure is done pid=12 2023-02-13 21:59:51,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:51,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6bd825938f619ef4084c64b1f66ce2a6, NAME => 'restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6.', STARTKEY => '', ENDKEY => ''} 2023-02-13 21:59:51,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table restartTableTwo 6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:51,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:51,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:51,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:51,461 INFO [StoreOpener-6bd825938f619ef4084c64b1f66ce2a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family of region 6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:51,463 DEBUG [StoreOpener-6bd825938f619ef4084c64b1f66ce2a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableTwo/6bd825938f619ef4084c64b1f66ce2a6/family 2023-02-13 21:59:51,463 DEBUG [StoreOpener-6bd825938f619ef4084c64b1f66ce2a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableTwo/6bd825938f619ef4084c64b1f66ce2a6/family 2023-02-13 21:59:51,464 INFO [StoreOpener-6bd825938f619ef4084c64b1f66ce2a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6bd825938f619ef4084c64b1f66ce2a6 columnFamilyName family 2023-02-13 21:59:51,465 INFO [StoreOpener-6bd825938f619ef4084c64b1f66ce2a6-1] regionserver.HStore(310): Store=6bd825938f619ef4084c64b1f66ce2a6/family, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:51,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableTwo/6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:51,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableTwo/6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:51,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:51,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableTwo/6bd825938f619ef4084c64b1f66ce2a6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 21:59:51,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 6bd825938f619ef4084c64b1f66ce2a6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67391690, jitterRate=0.004214435815811157}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 21:59:51,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 6bd825938f619ef4084c64b1f66ce2a6: Running coprocessor pre-open hook at 1676325591459Writing region info on filesystem at 1676325591459Initializing all the Stores at 1676325591461 (+2 ms)Instantiating store for column family {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325591461Cleaning up temporary data from old regions at 1676325591470 (+9 ms)Cleaning up detritus from prior splits at 1676325591470Running coprocessor post-open hooks at 1676325591477 (+7 ms)Region opened successfully at 1676325591477 2023-02-13 21:59:51,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6., pid=14, masterSystemTime=1676325591452 2023-02-13 21:59:51,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:51,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:51,483 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=6bd825938f619ef4084c64b1f66ce2a6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:51,484 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325591483"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325591483"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325591483"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325591483"}]},"ts":"1676325591483"} 2023-02-13 21:59:51,491 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-13 21:59:51,491 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 6bd825938f619ef4084c64b1f66ce2a6, server=jenkins-hbase20.apache.org,46497,1676325587650 in 188 msec 2023-02-13 21:59:51,494 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-13 21:59:51,496 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=restartTableTwo, region=6bd825938f619ef4084c64b1f66ce2a6, ASSIGN in 352 msec 2023-02-13 21:59:51,497 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=restartTableTwo execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 21:59:51,497 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"restartTableTwo","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325591497"}]},"ts":"1676325591497"} 2023-02-13 21:59:51,500 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=restartTableTwo, state=ENABLED in hbase:meta 2023-02-13 21:59:51,502 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=restartTableTwo execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 21:59:51,505 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=restartTableTwo in 435 msec 2023-02-13 21:59:51,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.MasterRpcServices(1227): Checking to see if procedure is done pid=12 2023-02-13 21:59:51,678 INFO [Listener at localhost.localdomain/34171] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:restartTableTwo, procId: 12 completed 2023-02-13 21:59:51,678 DEBUG [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(3430): Waiting until all regions of table restartTableTwo get assigned. Timeout = 60000ms 2023-02-13 21:59:51,679 INFO [Listener at localhost.localdomain/34171] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 21:59:51,682 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(3484): All regions for table restartTableTwo assigned to meta. Checking AM states. 2023-02-13 21:59:51,683 INFO [Listener at localhost.localdomain/34171] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 21:59:51,683 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(3504): All regions for table restartTableTwo assigned. 2023-02-13 21:59:51,685 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.HMaster$4(2102): Client=jenkins//148.251.75.209 create 'restartTableThree', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 21:59:51,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=restartTableThree 2023-02-13 21:59:51,688 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=restartTableThree execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 21:59:51,689 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.MasterRpcServices(697): Client=jenkins//148.251.75.209 procedure request for creating table: namespace: "default" qualifier: "restartTableThree" procId is: 15 2023-02-13 21:59:51,690 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=restartTableThree execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 21:59:51,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.MasterRpcServices(1227): Checking to see if procedure is done pid=15 2023-02-13 21:59:51,692 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/default/restartTableThree/6a1ed9b3fbb3a5a337d9be9c45e6e831 2023-02-13 21:59:51,693 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/default/restartTableThree/6a1ed9b3fbb3a5a337d9be9c45e6e831 empty. 2023-02-13 21:59:51,694 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/default/restartTableThree/6a1ed9b3fbb3a5a337d9be9c45e6e831 2023-02-13 21:59:51,694 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived restartTableThree regions 2023-02-13 21:59:51,713 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp/data/default/restartTableThree/.tabledesc/.tableinfo.0000000001 2023-02-13 21:59:51,715 INFO [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(7671): creating {ENCODED => 6a1ed9b3fbb3a5a337d9be9c45e6e831, NAME => 'restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='restartTableThree', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.tmp 2023-02-13 21:59:51,733 DEBUG [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(865): Instantiated restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:51,733 DEBUG [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(1603): Closing 6a1ed9b3fbb3a5a337d9be9c45e6e831, disabling compactions & flushes 2023-02-13 21:59:51,733 INFO [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(1625): Closing region restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831. 2023-02-13 21:59:51,733 DEBUG [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831. 2023-02-13 21:59:51,733 DEBUG [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(1713): Acquired close lock on restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831. after waiting 0 ms 2023-02-13 21:59:51,733 DEBUG [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(1723): Updates disabled for region restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831. 2023-02-13 21:59:51,733 INFO [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(1837): Closed restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831. 2023-02-13 21:59:51,734 DEBUG [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(1557): Region close journal for 6a1ed9b3fbb3a5a337d9be9c45e6e831: Waiting for close lock at 1676325591733Disabling compacts and flushes for region at 1676325591733Disabling writes for close at 1676325591733Writing region close event to WAL at 1676325591733Closed at 1676325591733 2023-02-13 21:59:51,737 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=restartTableThree execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 21:59:51,739 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831.","families":{"info":[{"qualifier":"regioninfo","vlen":51,"tag":[],"timestamp":"1676325591739"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325591739"}]},"ts":"1676325591739"} 2023-02-13 21:59:51,742 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 21:59:51,744 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=restartTableThree execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 21:59:51,744 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"restartTableThree","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325591744"}]},"ts":"1676325591744"} 2023-02-13 21:59:51,746 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=restartTableThree, state=ENABLING in hbase:meta 2023-02-13 21:59:51,751 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 21:59:51,752 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 21:59:51,752 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 21:59:51,752 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 21:59:51,752 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 21:59:51,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableThree, region=6a1ed9b3fbb3a5a337d9be9c45e6e831, ASSIGN}] 2023-02-13 21:59:51,755 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableThree, region=6a1ed9b3fbb3a5a337d9be9c45e6e831, ASSIGN 2023-02-13 21:59:51,756 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=restartTableThree, region=6a1ed9b3fbb3a5a337d9be9c45e6e831, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,45487,1676325587722; forceNewPlan=false, retain=false 2023-02-13 21:59:51,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.MasterRpcServices(1227): Checking to see if procedure is done pid=15 2023-02-13 21:59:51,907 INFO [jenkins-hbase20:43889] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 21:59:51,908 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=6a1ed9b3fbb3a5a337d9be9c45e6e831, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:51,908 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831.","families":{"info":[{"qualifier":"regioninfo","vlen":51,"tag":[],"timestamp":"1676325591908"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325591908"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325591908"}]},"ts":"1676325591908"} 2023-02-13 21:59:51,911 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure 6a1ed9b3fbb3a5a337d9be9c45e6e831, server=jenkins-hbase20.apache.org,45487,1676325587722}] 2023-02-13 21:59:51,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.MasterRpcServices(1227): Checking to see if procedure is done pid=15 2023-02-13 21:59:52,065 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:52,065 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 21:59:52,067 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:60580, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 21:59:52,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831. 2023-02-13 21:59:52,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6a1ed9b3fbb3a5a337d9be9c45e6e831, NAME => 'restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831.', STARTKEY => '', ENDKEY => ''} 2023-02-13 21:59:52,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table restartTableThree 6a1ed9b3fbb3a5a337d9be9c45e6e831 2023-02-13 21:59:52,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:52,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 6a1ed9b3fbb3a5a337d9be9c45e6e831 2023-02-13 21:59:52,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 6a1ed9b3fbb3a5a337d9be9c45e6e831 2023-02-13 21:59:52,075 INFO [StoreOpener-6a1ed9b3fbb3a5a337d9be9c45e6e831-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family of region 6a1ed9b3fbb3a5a337d9be9c45e6e831 2023-02-13 21:59:52,077 DEBUG [StoreOpener-6a1ed9b3fbb3a5a337d9be9c45e6e831-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableThree/6a1ed9b3fbb3a5a337d9be9c45e6e831/family 2023-02-13 21:59:52,077 DEBUG [StoreOpener-6a1ed9b3fbb3a5a337d9be9c45e6e831-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableThree/6a1ed9b3fbb3a5a337d9be9c45e6e831/family 2023-02-13 21:59:52,078 INFO [StoreOpener-6a1ed9b3fbb3a5a337d9be9c45e6e831-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6a1ed9b3fbb3a5a337d9be9c45e6e831 columnFamilyName family 2023-02-13 21:59:52,079 INFO [StoreOpener-6a1ed9b3fbb3a5a337d9be9c45e6e831-1] regionserver.HStore(310): Store=6a1ed9b3fbb3a5a337d9be9c45e6e831/family, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:52,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableThree/6a1ed9b3fbb3a5a337d9be9c45e6e831 2023-02-13 21:59:52,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableThree/6a1ed9b3fbb3a5a337d9be9c45e6e831 2023-02-13 21:59:52,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 6a1ed9b3fbb3a5a337d9be9c45e6e831 2023-02-13 21:59:52,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableThree/6a1ed9b3fbb3a5a337d9be9c45e6e831/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 21:59:52,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 6a1ed9b3fbb3a5a337d9be9c45e6e831; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73645433, jitterRate=0.09740246832370758}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 21:59:52,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 6a1ed9b3fbb3a5a337d9be9c45e6e831: Running coprocessor pre-open hook at 1676325592073Writing region info on filesystem at 1676325592073Initializing all the Stores at 1676325592075 (+2 ms)Instantiating store for column family {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325592075Cleaning up temporary data from old regions at 1676325592083 (+8 ms)Cleaning up detritus from prior splits at 1676325592084 (+1 ms)Running coprocessor post-open hooks at 1676325592094 (+10 ms)Region opened successfully at 1676325592094 2023-02-13 21:59:52,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831., pid=17, masterSystemTime=1676325592065 2023-02-13 21:59:52,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831. 2023-02-13 21:59:52,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831. 2023-02-13 21:59:52,100 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=6a1ed9b3fbb3a5a337d9be9c45e6e831, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:52,100 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831.","families":{"info":[{"qualifier":"regioninfo","vlen":51,"tag":[],"timestamp":"1676325592100"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325592100"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325592100"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325592100"}]},"ts":"1676325592100"} 2023-02-13 21:59:52,106 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-02-13 21:59:52,106 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure 6a1ed9b3fbb3a5a337d9be9c45e6e831, server=jenkins-hbase20.apache.org,45487,1676325587722 in 192 msec 2023-02-13 21:59:52,108 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-02-13 21:59:52,109 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=restartTableThree, region=6a1ed9b3fbb3a5a337d9be9c45e6e831, ASSIGN in 353 msec 2023-02-13 21:59:52,110 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=restartTableThree execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 21:59:52,111 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"restartTableThree","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325592110"}]},"ts":"1676325592110"} 2023-02-13 21:59:52,113 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=restartTableThree, state=ENABLED in hbase:meta 2023-02-13 21:59:52,115 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=restartTableThree execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 21:59:52,118 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; CreateTableProcedure table=restartTableThree in 430 msec 2023-02-13 21:59:52,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43889] master.MasterRpcServices(1227): Checking to see if procedure is done pid=15 2023-02-13 21:59:52,295 INFO [Listener at localhost.localdomain/34171] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:restartTableThree, procId: 15 completed 2023-02-13 21:59:52,296 DEBUG [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(3430): Waiting until all regions of table restartTableThree get assigned. Timeout = 60000ms 2023-02-13 21:59:52,296 INFO [Listener at localhost.localdomain/34171] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 21:59:52,300 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(3484): All regions for table restartTableThree assigned to meta. Checking AM states. 2023-02-13 21:59:52,301 INFO [Listener at localhost.localdomain/34171] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 21:59:52,301 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(3504): All regions for table restartTableThree assigned. 2023-02-13 21:59:52,301 INFO [Listener at localhost.localdomain/34171] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 21:59:52,308 INFO [Listener at localhost.localdomain/34171] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 21:59:52,313 INFO [Listener at localhost.localdomain/34171] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 21:59:52,317 INFO [Listener at localhost.localdomain/34171] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 21:59:52,318 INFO [Listener at localhost.localdomain/34171] master.SnapshotOfRegionAssignmentFromMeta(102): Start to scan the hbase:meta for the current region assignment snappshot 2023-02-13 21:59:52,326 INFO [Listener at localhost.localdomain/34171] master.SnapshotOfRegionAssignmentFromMeta(164): Finished to scan the hbase:meta for the current region assignmentsnapshot 2023-02-13 21:59:52,326 INFO [Listener at localhost.localdomain/34171] master.TestRetainAssignmentOnRestart(157): Stopping HMaster and jenkins-hbase20.apache.org,46497,1676325587650 server 2023-02-13 21:59:52,326 INFO [Listener at localhost.localdomain/34171] hbase.MiniHBaseCluster(631): Stopping Thread[M:0;jenkins-hbase20:43889,5,FailOnTimeoutGroup] 2023-02-13 21:59:52,326 INFO [Listener at localhost.localdomain/34171] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,43889,1676325586653' ***** 2023-02-13 21:59:52,326 INFO [Listener at localhost.localdomain/34171] regionserver.HRegionServer(2310): STOPPED: Stopping master 0 2023-02-13 21:59:52,327 DEBUG [M:0;jenkins-hbase20:43889] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@74a6e69a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 21:59:52,328 INFO [M:0;jenkins-hbase20:43889] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,43889,1676325586653 2023-02-13 21:59:52,329 DEBUG [M:0;jenkins-hbase20:43889] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6adefa11 to 127.0.0.1:57335 2023-02-13 21:59:52,329 DEBUG [M:0;jenkins-hbase20:43889] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 21:59:52,330 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:46497-0x1015dda45730001, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:52,330 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:52,330 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:52,330 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:52,331 INFO [Listener at localhost.localdomain/34171] hbase.LocalHBaseCluster(360): Waiting on jenkins-hbase20.apache.org,43889,1676325586653 2023-02-13 21:59:52,332 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:52,332 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 21:59:52,330 INFO [M:0;jenkins-hbase20:43889] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,43889,1676325586653; all regions closed. 2023-02-13 21:59:52,332 DEBUG [M:0;jenkins-hbase20:43889] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 21:59:52,332 DEBUG [M:0;jenkins-hbase20:43889] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 21:59:52,333 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 21:59:52,333 DEBUG [M:0;jenkins-hbase20:43889] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 21:59:52,333 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46497-0x1015dda45730001, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 21:59:52,333 INFO [M:0;jenkins-hbase20:43889] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 21:59:52,334 INFO [M:0;jenkins-hbase20:43889] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 21:59:52,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 21:59:52,335 INFO [M:0;jenkins-hbase20:43889] hbase.ChoreService(369): Chore service for: master/jenkins-hbase20:0 had [] on shutdown 2023-02-13 21:59:52,335 DEBUG [M:0;jenkins-hbase20:43889] master.HMaster(1502): Stopping service threads 2023-02-13 21:59:52,335 INFO [M:0;jenkins-hbase20:43889] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 21:59:52,335 INFO [M:0;jenkins-hbase20:43889] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 21:59:52,333 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676325588844] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676325588844,5,FailOnTimeoutGroup] 2023-02-13 21:59:52,333 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676325588843] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676325588843,5,FailOnTimeoutGroup] 2023-02-13 21:59:52,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 21:59:52,336 ERROR [M:0;jenkins-hbase20:43889] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] 2023-02-13 21:59:52,337 INFO [M:0;jenkins-hbase20:43889] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 21:59:52,337 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 21:59:52,338 DEBUG [M:0;jenkins-hbase20:43889] zookeeper.ZKUtil(398): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 21:59:52,338 WARN [M:0;jenkins-hbase20:43889] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 21:59:52,338 INFO [M:0;jenkins-hbase20:43889] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 21:59:52,339 INFO [M:0;jenkins-hbase20:43889] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 21:59:52,340 DEBUG [M:0;jenkins-hbase20:43889] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 21:59:52,340 INFO [M:0;jenkins-hbase20:43889] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:52,340 DEBUG [M:0;jenkins-hbase20:43889] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:52,340 DEBUG [M:0;jenkins-hbase20:43889] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 21:59:52,340 DEBUG [M:0;jenkins-hbase20:43889] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:52,341 INFO [M:0;jenkins-hbase20:43889] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=64.23 KB heapSize=76.20 KB 2023-02-13 21:59:52,407 INFO [M:0;jenkins-hbase20:43889] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=64.23 KB at sequenceid=141 (bloomFilter=true), to=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b150f663144a42b3b46719a7a873eb10 2023-02-13 21:59:52,448 DEBUG [M:0;jenkins-hbase20:43889] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b150f663144a42b3b46719a7a873eb10 as hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b150f663144a42b3b46719a7a873eb10 2023-02-13 21:59:52,458 INFO [M:0;jenkins-hbase20:43889] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b150f663144a42b3b46719a7a873eb10, entries=17, sequenceid=141, filesize=8.5 K 2023-02-13 21:59:52,460 INFO [M:0;jenkins-hbase20:43889] regionserver.HRegion(2947): Finished flush of dataSize ~64.23 KB/65775, heapSize ~76.19 KB/78016, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 119ms, sequenceid=141, compaction requested=false 2023-02-13 21:59:52,464 INFO [M:0;jenkins-hbase20:43889] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:52,465 DEBUG [M:0;jenkins-hbase20:43889] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676325592339Disabling compacts and flushes for region at 1676325592339Disabling writes for close at 1676325592340 (+1 ms)Obtaining lock to block concurrent updates at 1676325592341 (+1 ms)Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676325592341Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=65775, getHeapSize=78016, getOffHeapSize=0, getCellsCount=158 at 1676325592352 (+11 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676325592353 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676325592355 (+2 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676325592379 (+24 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676325592382 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676325592450 (+68 ms)Finished flush of dataSize ~64.23 KB/65775, heapSize ~76.19 KB/78016, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 119ms, sequenceid=141, compaction requested=false at 1676325592461 (+11 ms)Writing region close event to WAL at 1676325592464 (+3 ms)Closed at 1676325592464 2023-02-13 21:59:52,472 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 21:59:52,472 INFO [M:0;jenkins-hbase20:43889] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 21:59:52,474 INFO [M:0;jenkins-hbase20:43889] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:43889 2023-02-13 21:59:52,485 DEBUG [M:0;jenkins-hbase20:43889] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase20.apache.org,43889,1676325586653 already deleted, retry=false 2023-02-13 21:59:52,510 WARN [RS-EventLoopGroup-5-1] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase20.apache.org/148.251.75.209:43889 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase20.apache.org/148.251.75.209:43889 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 21:59:52,511 DEBUG [RS-EventLoopGroup-5-1] ipc.FailedServers(52): Added failed server with address jenkins-hbase20.apache.org/148.251.75.209:43889 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase20.apache.org/148.251.75.209:43889 2023-02-13 21:59:52,511 DEBUG [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(2737): No master found; retry 2023-02-13 21:59:52,554 WARN [RS-EventLoopGroup-5-3] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase20.apache.org/148.251.75.209:43889 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase20.apache.org/148.251.75.209:43889 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 21:59:52,554 DEBUG [RS-EventLoopGroup-5-3] ipc.FailedServers(52): Added failed server with address jenkins-hbase20.apache.org/148.251.75.209:43889 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase20.apache.org/148.251.75.209:43889 2023-02-13 21:59:52,555 DEBUG [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(2737): No master found; retry 2023-02-13 21:59:52,563 WARN [RS-EventLoopGroup-5-2] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase20.apache.org/148.251.75.209:43889 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase20.apache.org/148.251.75.209:43889 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 21:59:52,563 DEBUG [RS-EventLoopGroup-5-2] ipc.FailedServers(52): Added failed server with address jenkins-hbase20.apache.org/148.251.75.209:43889 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase20.apache.org/148.251.75.209:43889 2023-02-13 21:59:52,563 DEBUG [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(2737): No master found; retry 2023-02-13 21:59:52,587 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 21:59:52,588 INFO [M:0;jenkins-hbase20:43889] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,43889,1676325586653; zookeeper connection closed. 2023-02-13 21:59:52,588 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@25f25783 rejected from java.util.concurrent.ThreadPoolExecutor@4de62f4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 25] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:52,589 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:43889-0x1015dda45730000, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 21:59:52,589 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5f9d37eb rejected from java.util.concurrent.ThreadPoolExecutor@4de62f4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 25] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:52,590 INFO [Listener at localhost.localdomain/34171] hbase.MiniHBaseCluster(508): Stopping Thread[RS:0;jenkins-hbase20:46497,5,FailOnTimeoutGroup] 2023-02-13 21:59:52,590 INFO [Listener at localhost.localdomain/34171] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,46497,1676325587650' ***** 2023-02-13 21:59:52,590 INFO [Listener at localhost.localdomain/34171] regionserver.HRegionServer(2310): STOPPED: Stopping rs 0 2023-02-13 21:59:52,590 INFO [Listener at localhost.localdomain/34171] master.TestRetainAssignmentOnRestart(163): Sleeping a bit 2023-02-13 21:59:52,663 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 21:59:52,711 INFO [RS:0;jenkins-hbase20:46497] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 21:59:52,712 INFO [RS:0;jenkins-hbase20:46497] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 21:59:52,712 INFO [RS:0;jenkins-hbase20:46497] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 21:59:52,712 INFO [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(3304): Received CLOSE for 6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:52,713 INFO [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:52,714 DEBUG [RS:0;jenkins-hbase20:46497] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7c316a08 to 127.0.0.1:57335 2023-02-13 21:59:52,714 DEBUG [RS:0;jenkins-hbase20:46497] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 21:59:52,714 INFO [RS:0;jenkins-hbase20:46497] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 21:59:52,714 INFO [RS:0;jenkins-hbase20:46497] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 21:59:52,714 INFO [RS:0;jenkins-hbase20:46497] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 21:59:52,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 6bd825938f619ef4084c64b1f66ce2a6, disabling compactions & flushes 2023-02-13 21:59:52,715 INFO [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 21:59:52,715 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:52,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:52,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. after waiting 0 ms 2023-02-13 21:59:52,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:52,715 INFO [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-02-13 21:59:52,715 DEBUG [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(1479): Online Regions={6bd825938f619ef4084c64b1f66ce2a6=restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6., 1588230740=hbase:meta,,1.1588230740} 2023-02-13 21:59:52,716 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 21:59:52,717 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 21:59:52,717 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 21:59:52,717 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 21:59:52,717 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 21:59:52,717 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=5.15 KB heapSize=9.42 KB 2023-02-13 21:59:52,718 DEBUG [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(1505): Waiting on 1588230740, 6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:52,724 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableTwo/6bd825938f619ef4084c64b1f66ce2a6/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 21:59:52,725 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:52,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 6bd825938f619ef4084c64b1f66ce2a6: Waiting for close lock at 1676325592714Running coprocessor pre-close hooks at 1676325592714Disabling compacts and flushes for region at 1676325592714Disabling writes for close at 1676325592715 (+1 ms)Writing region close event to WAL at 1676325592716 (+1 ms)Running coprocessor post-close hooks at 1676325592725 (+9 ms)Closed at 1676325592725 2023-02-13 21:59:52,726 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:52,741 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.78 KB at sequenceid=24 (bloomFilter=false), to=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/.tmp/info/6a18bfdc8eb34c5d8573fb9b6fd5ae2f 2023-02-13 21:59:52,763 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 21:59:52,771 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=380 B at sequenceid=24 (bloomFilter=false), to=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/.tmp/table/f2d5cd5ed4a24573ae8fb155e3fcfe35 2023-02-13 21:59:52,781 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/.tmp/info/6a18bfdc8eb34c5d8573fb9b6fd5ae2f as hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/info/6a18bfdc8eb34c5d8573fb9b6fd5ae2f 2023-02-13 21:59:52,790 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/info/6a18bfdc8eb34c5d8573fb9b6fd5ae2f, entries=40, sequenceid=24, filesize=9.5 K 2023-02-13 21:59:52,792 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/.tmp/table/f2d5cd5ed4a24573ae8fb155e3fcfe35 as hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/table/f2d5cd5ed4a24573ae8fb155e3fcfe35 2023-02-13 21:59:52,800 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/table/f2d5cd5ed4a24573ae8fb155e3fcfe35, entries=8, sequenceid=24, filesize=5.0 K 2023-02-13 21:59:52,802 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~5.15 KB/5270, heapSize ~9.14 KB/9360, currentSize=0 B/0 for 1588230740 in 85ms, sequenceid=24, compaction requested=false 2023-02-13 21:59:52,804 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-13 21:59:52,812 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/recovered.edits/27.seqid, newMaxSeqId=27, maxSeqId=1 2023-02-13 21:59:52,813 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 21:59:52,813 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 21:59:52,813 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676325592716Running coprocessor pre-close hooks at 1676325592716Disabling compacts and flushes for region at 1676325592716Disabling writes for close at 1676325592717 (+1 ms)Obtaining lock to block concurrent updates at 1676325592717Preparing flush snapshotting stores in 1588230740 at 1676325592717Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=5270, getHeapSize=9600, getOffHeapSize=0, getCellsCount=48 at 1676325592718 (+1 ms)Flushing stores of hbase:meta,,1.1588230740 at 1676325592718Flushing 1588230740/info: creating writer at 1676325592719 (+1 ms)Flushing 1588230740/info: appending metadata at 1676325592725 (+6 ms)Flushing 1588230740/info: closing flushed file at 1676325592725Flushing 1588230740/table: creating writer at 1676325592754 (+29 ms)Flushing 1588230740/table: appending metadata at 1676325592758 (+4 ms)Flushing 1588230740/table: closing flushed file at 1676325592758Flushing 1588230740/info: reopening flushed file at 1676325592782 (+24 ms)Flushing 1588230740/table: reopening flushed file at 1676325592793 (+11 ms)Finished flush of dataSize ~5.15 KB/5270, heapSize ~9.14 KB/9360, currentSize=0 B/0 for 1588230740 in 85ms, sequenceid=24, compaction requested=false at 1676325592802 (+9 ms)Writing region close event to WAL at 1676325592808 (+6 ms)Running coprocessor post-close hooks at 1676325592813 (+5 ms)Closed at 1676325592813 2023-02-13 21:59:52,813 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 21:59:52,919 INFO [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,46497,1676325587650; all regions closed. 2023-02-13 21:59:52,930 DEBUG [RS:0;jenkins-hbase20:46497] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/oldWALs 2023-02-13 21:59:52,930 INFO [RS:0;jenkins-hbase20:46497] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C46497%2C1676325587650.meta:.meta(num 1676325589363) 2023-02-13 21:59:52,937 DEBUG [RS:0;jenkins-hbase20:46497] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/oldWALs 2023-02-13 21:59:52,937 INFO [RS:0;jenkins-hbase20:46497] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C46497%2C1676325587650:(num 1676325589217) 2023-02-13 21:59:52,937 DEBUG [RS:0;jenkins-hbase20:46497] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 21:59:52,937 INFO [RS:0;jenkins-hbase20:46497] regionserver.LeaseManager(133): Closed leases 2023-02-13 21:59:52,937 INFO [RS:0;jenkins-hbase20:46497] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 21:59:52,938 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 21:59:52,939 INFO [RS:0;jenkins-hbase20:46497] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:46497 2023-02-13 21:59:52,943 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:52,943 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:46497-0x1015dda45730001, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:52,943 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:52,943 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@735ebce3 rejected from java.util.concurrent.ThreadPoolExecutor@77613650[Shutting down, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:52,943 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 21:59:52,943 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:46497-0x1015dda45730001, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 21:59:52,943 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 21:59:52,943 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5249de9 rejected from java.util.concurrent.ThreadPoolExecutor@77613650[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:52,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:52,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:52,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:52,948 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase20.apache.org,46497,1676325587650 znode expired, triggering replicatorRemoved event 2023-02-13 21:59:52,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:52,949 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase20.apache.org,46497,1676325587650 znode expired, triggering replicatorRemoved event 2023-02-13 21:59:52,952 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:52,952 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:52,952 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:52,952 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:53,047 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:46497-0x1015dda45730001, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 21:59:53,047 INFO [RS:0;jenkins-hbase20:46497] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,46497,1676325587650; zookeeper connection closed. 2023-02-13 21:59:53,047 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7db6f58f rejected from java.util.concurrent.ThreadPoolExecutor@77613650[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:53,047 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@24a7d26a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@24a7d26a 2023-02-13 21:59:53,047 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:46497-0x1015dda45730001, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 21:59:53,047 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@73a64de2 rejected from java.util.concurrent.ThreadPoolExecutor@77613650[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:53,559 DEBUG [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(2737): No master found; retry 2023-02-13 21:59:53,568 DEBUG [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(2737): No master found; retry 2023-02-13 21:59:54,563 DEBUG [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(2737): No master found; retry 2023-02-13 21:59:54,572 DEBUG [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(2737): No master found; retry 2023-02-13 21:59:54,590 INFO [Listener at localhost.localdomain/34171] master.TestRetainAssignmentOnRestart(166): Starting HMaster and region server jenkins-hbase20.apache.org,46497,1676325587650 second time with the same port 2023-02-13 21:59:54,602 INFO [Listener at localhost.localdomain/34171] client.ConnectionUtils(127): master/jenkins-hbase20:0 server-side Connection retries=45 2023-02-13 21:59:54,602 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:54,602 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:54,603 INFO [Listener at localhost.localdomain/34171] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 21:59:54,603 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:54,603 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 21:59:54,603 INFO [Listener at localhost.localdomain/34171] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 21:59:54,606 INFO [Listener at localhost.localdomain/34171] ipc.NettyRpcServer(120): Bind to /148.251.75.209:46045 2023-02-13 21:59:54,607 INFO [Listener at localhost.localdomain/34171] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:54,608 INFO [Listener at localhost.localdomain/34171] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:54,609 INFO [Listener at localhost.localdomain/34171] zookeeper.RecoverableZooKeeper(93): Process identifier=master:46045 connecting to ZooKeeper ensemble=127.0.0.1:57335 2023-02-13 21:59:54,612 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:460450x0, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 21:59:54,614 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:46045-0x1015dda45730009 connected 2023-02-13 21:59:54,617 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(164): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 21:59:54,617 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(162): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/running 2023-02-13 21:59:54,618 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(164): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 21:59:54,623 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46045 2023-02-13 21:59:54,625 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46045 2023-02-13 21:59:54,625 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46045 2023-02-13 21:59:54,627 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46045 2023-02-13 21:59:54,627 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46045 2023-02-13 21:59:54,627 INFO [Listener at localhost.localdomain/34171] master.HMaster(439): hbase.rootdir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f, hbase.cluster.distributed=false 2023-02-13 21:59:54,629 DEBUG [pool-218-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: INIT 2023-02-13 21:59:54,631 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase20.apache.org,46045,1676325594602 2023-02-13 21:59:54,633 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 21:59:54,633 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase20.apache.org,46045,1676325594602 2023-02-13 21:59:54,634 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:54,634 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:54,634 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:54,634 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:54,635 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 21:59:54,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 21:59:54,636 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase20.apache.org,46045,1676325594602 from backup master directory 2023-02-13 21:59:54,636 INFO [M:0;jenkins-hbase20:46045] regionserver.HRegionServer(952): ClusterId : 611d7ab4-6c3d-48dd-a873-3bff225d9a89 2023-02-13 21:59:54,637 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase20.apache.org,46045,1676325594602 2023-02-13 21:59:54,638 WARN [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 21:59:54,638 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 21:59:54,638 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase20.apache.org,46045,1676325594602 2023-02-13 21:59:54,651 INFO [Listener at localhost.localdomain/34171] client.ConnectionUtils(127): regionserver/jenkins-hbase20:46497 server-side Connection retries=45 2023-02-13 21:59:54,652 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:54,652 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:54,652 INFO [Listener at localhost.localdomain/34171] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 21:59:54,652 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:54,652 INFO [Listener at localhost.localdomain/34171] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 21:59:54,653 INFO [Listener at localhost.localdomain/34171] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 21:59:54,656 INFO [Listener at localhost.localdomain/34171] ipc.NettyRpcServer(120): Bind to /148.251.75.209:46497 2023-02-13 21:59:54,657 INFO [Listener at localhost.localdomain/34171] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 21:59:54,662 DEBUG [Listener at localhost.localdomain/34171] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 21:59:54,663 INFO [Listener at localhost.localdomain/34171] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:54,664 INFO [Listener at localhost.localdomain/34171] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:54,665 INFO [Listener at localhost.localdomain/34171] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46497 connecting to ZooKeeper ensemble=127.0.0.1:57335 2023-02-13 21:59:54,668 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:464970x0, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 21:59:54,670 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(162): regionserver:464970x0, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 21:59:54,670 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(162): regionserver:464970x0, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/running 2023-02-13 21:59:54,671 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:46497-0x1015dda4573000a connected 2023-02-13 21:59:54,672 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ZKUtil(164): regionserver:46497-0x1015dda4573000a, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 21:59:54,673 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46497 2023-02-13 21:59:54,673 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46497 2023-02-13 21:59:54,673 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46497 2023-02-13 21:59:54,674 INFO [master/jenkins-hbase20:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:54,680 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46497 2023-02-13 21:59:54,680 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46497 2023-02-13 21:59:54,685 INFO [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(952): ClusterId : 611d7ab4-6c3d-48dd-a873-3bff225d9a89 2023-02-13 21:59:54,686 DEBUG [RS:3;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 21:59:54,688 DEBUG [RS:3;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 21:59:54,688 DEBUG [RS:3;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 21:59:54,689 DEBUG [RS:3;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 21:59:54,691 DEBUG [RS:3;jenkins-hbase20:46497] zookeeper.ReadOnlyZKClient(139): Connect 0x0393332b to 127.0.0.1:57335 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 21:59:54,697 DEBUG [RS:3;jenkins-hbase20:46497] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@d54dd29, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 21:59:54,698 DEBUG [RS:3;jenkins-hbase20:46497] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@96b585b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 21:59:54,707 DEBUG [RS:3;jenkins-hbase20:46497] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase20:46497 2023-02-13 21:59:54,707 INFO [RS:3;jenkins-hbase20:46497] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 21:59:54,707 INFO [RS:3;jenkins-hbase20:46497] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 21:59:54,707 DEBUG [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 21:59:54,708 INFO [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,46045,1676325594602 with isa=jenkins-hbase20.apache.org/148.251.75.209:46497, startcode=1676325594651 2023-02-13 21:59:54,709 DEBUG [RS:3;jenkins-hbase20:46497] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 21:59:54,717 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42243, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 21:59:54,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46045] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 21:59:54,719 DEBUG [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 21:59:54,719 WARN [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-13 21:59:54,730 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x3e78901a to 127.0.0.1:57335 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 21:59:54,734 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6492c78a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 21:59:54,734 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 21:59:54,735 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 21:59:54,735 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 21:59:54,744 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/WALs/jenkins-hbase20.apache.org,43889,1676325586653 to hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/WALs/jenkins-hbase20.apache.org,43889,1676325586653-dead as it is dead 2023-02-13 21:59:54,746 INFO [master/jenkins-hbase20:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/WALs/jenkins-hbase20.apache.org,43889,1676325586653-dead/jenkins-hbase20.apache.org%2C43889%2C1676325586653.1676325588207 2023-02-13 21:59:54,750 INFO [master/jenkins-hbase20:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/WALs/jenkins-hbase20.apache.org,43889,1676325586653-dead/jenkins-hbase20.apache.org%2C43889%2C1676325586653.1676325588207 after 4ms 2023-02-13 21:59:54,752 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/WALs/jenkins-hbase20.apache.org,43889,1676325586653-dead/jenkins-hbase20.apache.org%2C43889%2C1676325586653.1676325588207 to hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase20.apache.org%2C43889%2C1676325586653.1676325588207 2023-02-13 21:59:54,752 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/WALs/jenkins-hbase20.apache.org,43889,1676325586653-dead 2023-02-13 21:59:54,752 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/WALs/jenkins-hbase20.apache.org,46045,1676325594602 2023-02-13 21:59:54,756 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C46045%2C1676325594602, suffix=, logDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/WALs/jenkins-hbase20.apache.org,46045,1676325594602, archiveDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/oldWALs, maxLogs=10 2023-02-13 21:59:54,771 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK] 2023-02-13 21:59:54,774 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK] 2023-02-13 21:59:54,774 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK] 2023-02-13 21:59:54,781 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/WALs/jenkins-hbase20.apache.org,46045,1676325594602/jenkins-hbase20.apache.org%2C46045%2C1676325594602.1676325594756 2023-02-13 21:59:54,782 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK], DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK], DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK]] 2023-02-13 21:59:54,782 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 21:59:54,782 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:54,782 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:54,782 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:54,786 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:54,787 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 21:59:54,788 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 21:59:54,797 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b150f663144a42b3b46719a7a873eb10 2023-02-13 21:59:54,799 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:54,801 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5178): Found 1 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-02-13 21:59:54,802 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5275): Replaying edits from hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase20.apache.org%2C43889%2C1676325586653.1676325588207 2023-02-13 21:59:54,820 INFO [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,46045,1676325594602 with isa=jenkins-hbase20.apache.org/148.251.75.209:46497, startcode=1676325594651 2023-02-13 21:59:54,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46045] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 21:59:54,821 DEBUG [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 21:59:54,821 WARN [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 200 ms and then retrying. 2023-02-13 21:59:54,825 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5463): Applied 0, skipped 160, firstSequenceIdInLog=3, maxSequenceIdInLog=143, path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase20.apache.org%2C43889%2C1676325586653.1676325588207 2023-02-13 21:59:54,827 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5085): Deleted recovered.edits file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase20.apache.org%2C43889%2C1676325586653.1676325588207 2023-02-13 21:59:54,832 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:54,835 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/143.seqid, newMaxSeqId=143, maxSeqId=1 2023-02-13 21:59:54,837 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=144; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69625611, jitterRate=0.03750245273113251}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 21:59:54,837 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676325594782Initializing all the Stores at 1676325594784 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325594784Cleaning up temporary data from old regions at 1676325594828 (+44 ms)Cleaning up detritus from prior splits at 1676325594829 (+1 ms)Region opened successfully at 1676325594837 (+8 ms) 2023-02-13 21:59:54,837 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 21:59:54,838 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-13 21:59:54,838 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-13 21:59:54,839 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 21:59:54,839 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 21:59:54,857 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 21:59:54,858 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 21:59:54,858 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 21:59:54,858 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 21:59:54,859 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=restartTableOne 2023-02-13 21:59:54,859 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=restartTableTwo 2023-02-13 21:59:54,859 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; CreateTableProcedure table=restartTableThree 2023-02-13 21:59:54,860 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 20 msec 2023-02-13 21:59:54,860 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 21:59:54,861 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 21:59:54,861 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,46497,1676325587650, table=hbase:meta, region=1588230740 2023-02-13 21:59:54,863 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 3 possibly 'live' servers, and 0 'splitting'. 2023-02-13 21:59:54,865 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:54,865 DEBUG [RS:2;jenkins-hbase20:45487] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 21:59:54,866 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:54,868 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,46497,1676325587650 already deleted, retry=false 2023-02-13 21:59:54,868 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase20.apache.org,46497,1676325587650 on jenkins-hbase20.apache.org,46045,1676325594602 2023-02-13 21:59:54,869 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:44291, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 21:59:54,869 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=46045] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:545) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15954) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 21:59:54,874 DEBUG [RS:1;jenkins-hbase20:34119] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 21:59:54,876 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56513, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 21:59:54,876 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase20.apache.org,46497,1676325587650, splitWal=true, meta=true 2023-02-13 21:59:54,876 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=46045] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:545) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15954) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 21:59:54,876 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1736): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase20.apache.org,46497,1676325587650 (carryingMeta=true) jenkins-hbase20.apache.org,46497,1676325587650/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@3766c813[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-02-13 21:59:54,886 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 21:59:54,886 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 21:59:54,887 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/balancer 2023-02-13 21:59:54,887 INFO [master/jenkins-hbase20:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 21:59:54,888 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 21:59:54,889 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 21:59:54,889 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 21:59:54,890 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 21:59:54,890 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase20.apache.org,46045,1676325594602, sessionid=0x1015dda45730009, setting cluster-up flag (Was=true) 2023-02-13 21:59:54,892 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 21:59:54,901 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,46045,1676325594602 2023-02-13 21:59:54,906 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 21:59:54,907 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,46045,1676325594602 2023-02-13 21:59:54,908 WARN [master/jenkins-hbase20:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/.hbase-snapshot/.tmp 2023-02-13 21:59:54,909 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 21:59:54,909 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 21:59:54,909 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 21:59:54,909 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 21:59:54,909 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase20:0, corePoolSize=10, maxPoolSize=10 2023-02-13 21:59:54,909 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:54,909 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-13 21:59:54,909 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:54,915 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325624915 2023-02-13 21:59:54,915 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 21:59:54,916 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 21:59:54,916 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 21:59:54,916 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 21:59:54,916 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 21:59:54,917 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 21:59:54,917 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:54,918 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 21:59:54,918 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 21:59:54,918 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 21:59:54,920 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 21:59:54,920 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase20.apache.org,46497,1676325587650; numProcessing=1 2023-02-13 21:59:54,921 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 21:59:54,921 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase20.apache.org,46497,1676325587650, splitWal=true, meta=true 2023-02-13 21:59:54,922 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676325594921,5,FailOnTimeoutGroup] 2023-02-13 21:59:54,922 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676325594922,5,FailOnTimeoutGroup] 2023-02-13 21:59:54,922 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:54,923 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 21:59:54,923 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:54,923 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:54,923 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676325594633, completionTime=-1 2023-02-13 21:59:54,924 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=2; waited=0ms, expecting min=3 server(s), max=3 server(s), timeout=4500ms, lastChange=0ms 2023-02-13 21:59:54,924 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase20.apache.org,46497,1676325587650, splitWal=true, meta=true, isMeta: true 2023-02-13 21:59:54,926 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,46497,1676325587650-splitting 2023-02-13 21:59:54,928 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,46497,1676325587650-splitting dir is empty, no logs to split. 2023-02-13 21:59:54,928 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase20.apache.org,46497,1676325587650 WAL count=0, meta=true 2023-02-13 21:59:54,932 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,46497,1676325587650-splitting dir is empty, no logs to split. 2023-02-13 21:59:54,932 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase20.apache.org,46497,1676325587650 WAL count=0, meta=true 2023-02-13 21:59:54,932 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase20.apache.org,46497,1676325587650 WAL splitting is done? wals=0, meta=true 2023-02-13 21:59:54,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 21:59:54,936 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 21:59:54,938 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 21:59:54,938 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=jenkins-hbase20.apache.org,46497,1676325587650; forceNewPlan=false, retain=false 2023-02-13 21:59:55,009 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'restartTableThree' 2023-02-13 21:59:55,010 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-13 21:59:55,011 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'restartTableOne' 2023-02-13 21:59:55,022 INFO [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,46045,1676325594602 with isa=jenkins-hbase20.apache.org/148.251.75.209:46497, startcode=1676325594651 2023-02-13 21:59:55,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46045] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:55,025 DEBUG [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f 2023-02-13 21:59:55,025 DEBUG [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:36521 2023-02-13 21:59:55,025 DEBUG [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 21:59:55,026 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 21:59:55,026 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 21:59:55,026 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 21:59:55,027 DEBUG [RS:3;jenkins-hbase20:46497] zookeeper.ZKUtil(162): regionserver:46497-0x1015dda4573000a, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:55,027 WARN [RS:3;jenkins-hbase20:46497] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 21:59:55,027 INFO [RS:3;jenkins-hbase20:46497] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 21:59:55,027 DEBUG [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:55,027 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:55,027 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:55,028 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:55,028 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:55,028 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:55,028 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:55,032 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,46497,1676325594651] 2023-02-13 21:59:55,036 DEBUG [RS:3;jenkins-hbase20:46497] zookeeper.ZKUtil(162): regionserver:46497-0x1015dda4573000a, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:55,037 DEBUG [RS:3;jenkins-hbase20:46497] zookeeper.ZKUtil(162): regionserver:46497-0x1015dda4573000a, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:55,037 DEBUG [RS:3;jenkins-hbase20:46497] zookeeper.ZKUtil(162): regionserver:46497-0x1015dda4573000a, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:55,038 DEBUG [RS:3;jenkins-hbase20:46497] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 21:59:55,038 INFO [RS:3;jenkins-hbase20:46497] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 21:59:55,040 INFO [RS:3;jenkins-hbase20:46497] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 21:59:55,040 INFO [RS:3;jenkins-hbase20:46497] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 21:59:55,040 INFO [RS:3;jenkins-hbase20:46497] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:55,040 INFO [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 21:59:55,042 INFO [RS:3;jenkins-hbase20:46497] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:55,042 DEBUG [RS:3;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:46497, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:55,042 DEBUG [RS:3;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:46497, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:55,042 DEBUG [RS:3;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:46497, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:55,042 DEBUG [RS:3;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:46497, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:55,042 DEBUG [RS:3;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:46497, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:55,043 DEBUG [RS:3;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:46497, corePoolSize=2, maxPoolSize=2 2023-02-13 21:59:55,043 DEBUG [RS:3;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:46497, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:55,043 DEBUG [RS:3;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:46497, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:55,043 DEBUG [RS:3;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:46497, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:55,043 DEBUG [RS:3;jenkins-hbase20:46497] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:46497, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:55,044 INFO [RS:3;jenkins-hbase20:46497] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:55,044 INFO [RS:3;jenkins-hbase20:46497] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:55,044 INFO [RS:3;jenkins-hbase20:46497] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:55,056 INFO [RS:3;jenkins-hbase20:46497] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 21:59:55,056 INFO [RS:3;jenkins-hbase20:46497] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,46497,1676325594651-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:55,066 INFO [RS:3;jenkins-hbase20:46497] regionserver.Replication(203): jenkins-hbase20.apache.org,46497,1676325594651 started 2023-02-13 21:59:55,066 INFO [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,46497,1676325594651, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:46497, sessionid=0x1015dda4573000a 2023-02-13 21:59:55,066 DEBUG [RS:3;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 21:59:55,066 DEBUG [RS:3;jenkins-hbase20:46497] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:55,066 DEBUG [RS:3;jenkins-hbase20:46497] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,46497,1676325594651' 2023-02-13 21:59:55,066 DEBUG [RS:3;jenkins-hbase20:46497] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 21:59:55,067 INFO [Listener at localhost.localdomain/34171] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 21:59:55,067 DEBUG [RS:3;jenkins-hbase20:46497] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 21:59:55,068 DEBUG [RS:3;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 21:59:55,068 DEBUG [RS:3;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 21:59:55,068 DEBUG [RS:3;jenkins-hbase20:46497] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:55,068 DEBUG [RS:3;jenkins-hbase20:46497] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,46497,1676325594651' 2023-02-13 21:59:55,068 DEBUG [RS:3;jenkins-hbase20:46497] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 21:59:55,068 DEBUG [RS:3;jenkins-hbase20:46497] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 21:59:55,069 DEBUG [RS:3;jenkins-hbase20:46497] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 21:59:55,069 INFO [RS:3;jenkins-hbase20:46497] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 21:59:55,069 INFO [RS:3;jenkins-hbase20:46497] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 21:59:55,070 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:60342, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 21:59:55,070 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46497] ipc.CallRunner(144): callId: 43 service: ClientService methodName: Get size: 90 connection: 148.251.75.209:60342 deadline: 1676325655070, exception=org.apache.hadoop.hbase.NotServingRegionException: hbase:meta,,1 is not online on jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:55,074 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=150ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-13 21:59:55,074 WARN [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1230): hbase:meta,,1.1588230740 is NOT online; state={1588230740 state=OPEN, ts=1676325594939, server=jenkins-hbase20.apache.org,46497,1676325587650}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-02-13 21:59:55,089 INFO [jenkins-hbase20:46045] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 21:59:55,090 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,46497,1676325594651, state=OPENING 2023-02-13 21:59:55,092 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 21:59:55,092 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 21:59:55,092 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,46497,1676325594651}] 2023-02-13 21:59:55,174 INFO [RS:3;jenkins-hbase20:46497] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C46497%2C1676325594651, suffix=, logDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,46497,1676325594651, archiveDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/oldWALs, maxLogs=32 2023-02-13 21:59:55,174 WARN [ReadOnlyZKClient-127.0.0.1:57335@0x6263f8ce] client.ZKConnectionRegistry(168): Meta region is in state OPENING 2023-02-13 21:59:55,176 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46497] ipc.CallRunner(144): callId: 44 service: ClientService methodName: Get size: 90 connection: 148.251.75.209:60342 deadline: 1676325655175, exception=org.apache.hadoop.hbase.NotServingRegionException: hbase:meta,,1 is not online on jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:55,197 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK] 2023-02-13 21:59:55,197 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK] 2023-02-13 21:59:55,197 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK] 2023-02-13 21:59:55,201 INFO [RS:3;jenkins-hbase20:46497] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,46497,1676325594651/jenkins-hbase20.apache.org%2C46497%2C1676325594651.1676325595176 2023-02-13 21:59:55,201 DEBUG [RS:3;jenkins-hbase20:46497] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK], DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK], DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK]] 2023-02-13 21:59:55,248 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:55,248 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 21:59:55,250 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:60356, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=AdminService 2023-02-13 21:59:55,255 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 21:59:55,255 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 21:59:55,258 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C46497%2C1676325594651.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,46497,1676325594651, archiveDir=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/oldWALs, maxLogs=32 2023-02-13 21:59:55,275 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK] 2023-02-13 21:59:55,275 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK] 2023-02-13 21:59:55,275 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK] 2023-02-13 21:59:55,280 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,46497,1676325594651/jenkins-hbase20.apache.org%2C46497%2C1676325594651.meta.1676325595260.meta 2023-02-13 21:59:55,281 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38109,DS-a82c7ac4-1a23-4a35-b9c0-04299bb14801,DISK], DatanodeInfoWithStorage[127.0.0.1:37281,DS-b1ab4caf-e9ee-42e2-ad79-9eb9058f2d92,DISK], DatanodeInfoWithStorage[127.0.0.1:40259,DS-22c25aa3-a398-4428-b14b-650d35a9a0f7,DISK]] 2023-02-13 21:59:55,281 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 21:59:55,281 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 21:59:55,281 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 21:59:55,282 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 21:59:55,282 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 21:59:55,282 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:55,282 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 21:59:55,282 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 21:59:55,285 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 21:59:55,287 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/info 2023-02-13 21:59:55,287 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/info 2023-02-13 21:59:55,288 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 21:59:55,301 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/info/6a18bfdc8eb34c5d8573fb9b6fd5ae2f 2023-02-13 21:59:55,302 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:55,302 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 21:59:55,304 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/rep_barrier 2023-02-13 21:59:55,304 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/rep_barrier 2023-02-13 21:59:55,305 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 21:59:55,306 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:55,306 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 21:59:55,308 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/table 2023-02-13 21:59:55,308 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/table 2023-02-13 21:59:55,308 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 21:59:55,317 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/table/f2d5cd5ed4a24573ae8fb155e3fcfe35 2023-02-13 21:59:55,317 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:55,318 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740 2023-02-13 21:59:55,321 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740 2023-02-13 21:59:55,325 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 21:59:55,327 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 21:59:55,328 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=28; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70339615, jitterRate=0.04814194142818451}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 21:59:55,328 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676325595283Writing region info on filesystem at 1676325595283Initializing all the Stores at 1676325595284 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325595284Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325595284Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325595284Cleaning up temporary data from old regions at 1676325595322 (+38 ms)Cleaning up detritus from prior splits at 1676325595324 (+2 ms)Running coprocessor post-open hooks at 1676325595328 (+4 ms)Region opened successfully at 1676325595328 2023-02-13 21:59:55,329 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=20, masterSystemTime=1676325595248 2023-02-13 21:59:55,335 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 21:59:55,335 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:46497-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 21:59:55,336 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,46497,1676325594651, state=OPEN 2023-02-13 21:59:55,337 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 21:59:55,338 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 21:59:55,341 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-13 21:59:55,341 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,46497,1676325594651 in 246 msec 2023-02-13 21:59:55,344 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=18 2023-02-13 21:59:55,344 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 407 msec 2023-02-13 21:59:55,395 WARN [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase20.apache.org/148.251.75.209:43889 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: jenkins-hbase20.apache.org/148.251.75.209:43889 Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716) at org.apache.hbase.thirdparty.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:337) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 21:59:55,395 WARN [Listener at localhost.localdomain/34171] client.ConnectionImplementation(411): Checking master connection java.net.ConnectException: Call to address=jenkins-hbase20.apache.org/148.251.75.209:43889 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: jenkins-hbase20.apache.org/148.251.75.209:43889 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:186) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:386) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:417) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:413) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:99) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:195) at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$300(NettyRpcConnection.java:76) at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:296) at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:287) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:321) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:337) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: jenkins-hbase20.apache.org/148.251.75.209:43889 Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716) at org.apache.hbase.thirdparty.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:337) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 21:59:55,396 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.FailedServers(52): Added failed server with address jenkins-hbase20.apache.org/148.251.75.209:43889 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: jenkins-hbase20.apache.org/148.251.75.209:43889 2023-02-13 21:59:55,397 DEBUG [Listener at localhost.localdomain/34171] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase20.apache.org/148.251.75.209:43889 this server is in the failed servers list 2023-02-13 21:59:55,397 WARN [Listener at localhost.localdomain/34171] client.ConnectionImplementation(411): Checking master connection org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=jenkins-hbase20.apache.org/148.251.75.209:43889 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: jenkins-hbase20.apache.org/148.251.75.209:43889 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:236) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:386) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:417) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:413) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:430) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:326) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$200(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:582) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java) at org.apache.hadoop.hbase.client.ConnectionImplementation$MasterServiceState.isMasterRunning(ConnectionImplementation.java:1174) at org.apache.hadoop.hbase.client.ConnectionImplementation.lambda$masterConnectionStateSupplier$1(ConnectionImplementation.java:404) at org.apache.hadoop.hbase.client.ConnectionImplementation.isKeepAliveMasterConnectedAndRunning(ConnectionImplementation.java:1945) at org.apache.hadoop.hbase.client.ConnectionImplementation.getKeepAliveMasterService(ConnectionImplementation.java:1383) at org.apache.hadoop.hbase.client.ConnectionImplementation.getMaster(ConnectionImplementation.java:1373) at org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:57) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:102) at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:2954) at org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:542) at org.apache.hadoop.hbase.client.HTable.getDescriptor(HTable.java:228) at org.apache.hadoop.hbase.HBaseTestingUtility$9.evaluate(HBaseTestingUtility.java:3983) at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:183) at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:134) at org.apache.hadoop.hbase.HBaseCommonTestingUtility.waitFor(HBaseCommonTestingUtility.java:234) at org.apache.hadoop.hbase.HBaseTestingUtility.waitTableAvailable(HBaseTestingUtility.java:3136) at org.apache.hadoop.hbase.HBaseTestingUtility.waitTableAvailable(HBaseTestingUtility.java:3121) at org.apache.hadoop.hbase.master.TestRetainAssignmentOnRestart.testRetainAssignmentOnSingleRSRestart(TestRetainAssignmentOnRestart.java:176) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: jenkins-hbase20.apache.org/148.251.75.209:43889 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:350) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:427) ... 46 more 2023-02-13 21:59:55,398 DEBUG [Listener at localhost.localdomain/34171] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 21:59:55,400 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:34782, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 21:59:55,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46045] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.PleaseHoldException: Master is initializing at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2823) at org.apache.hadoop.hbase.master.MasterRpcServices.getTableDescriptors(MasterRpcServices.java:1073) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 21:59:55,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46045] ipc.CallRunner(144): callId: 50 service: MasterService methodName: GetTableDescriptors size: 61 connection: 148.251.75.209:34782 deadline: 1676325655404, exception=org.apache.hadoop.hbase.PleaseHoldException: Master is initializing 2023-02-13 21:59:55,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46045] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.PleaseHoldException: Master is initializing at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2823) at org.apache.hadoop.hbase.master.MasterRpcServices.getTableDescriptors(MasterRpcServices.java:1073) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 21:59:55,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46045] ipc.CallRunner(144): callId: 52 service: MasterService methodName: GetTableDescriptors size: 61 connection: 148.251.75.209:34782 deadline: 1676325655507, exception=org.apache.hadoop.hbase.PleaseHoldException: Master is initializing 2023-02-13 21:59:55,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46045] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.PleaseHoldException: Master is initializing at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2823) at org.apache.hadoop.hbase.master.MasterRpcServices.getTableDescriptors(MasterRpcServices.java:1073) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 21:59:55,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46045] ipc.CallRunner(144): callId: 54 service: MasterService methodName: GetTableDescriptors size: 61 connection: 148.251.75.209:34782 deadline: 1676325655710, exception=org.apache.hadoop.hbase.PleaseHoldException: Master is initializing 2023-02-13 21:59:56,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46045] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.PleaseHoldException: Master is initializing at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2823) at org.apache.hadoop.hbase.master.MasterRpcServices.getTableDescriptors(MasterRpcServices.java:1073) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 21:59:56,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46045] ipc.CallRunner(144): callId: 56 service: MasterService methodName: GetTableDescriptors size: 61 connection: 148.251.75.209:34782 deadline: 1676325656012, exception=org.apache.hadoop.hbase.PleaseHoldException: Master is initializing 2023-02-13 21:59:56,075 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 21:59:56,078 DEBUG [hconnection-0x55db01f3-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 21:59:56,080 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:60372, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-02-13 21:59:56,083 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=b2169e2475351a310eb3323519b34622, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,34119,1676325587690, regionLocation=jenkins-hbase20.apache.org,34119,1676325587690, openSeqNum=2 2023-02-13 21:59:56,083 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=364f7b221bf1134db4d8b7aca4f9c7a0, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,34119,1676325587690, regionLocation=jenkins-hbase20.apache.org,34119,1676325587690, openSeqNum=2 2023-02-13 21:59:56,084 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=6a1ed9b3fbb3a5a337d9be9c45e6e831, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45487,1676325587722, regionLocation=jenkins-hbase20.apache.org,45487,1676325587722, openSeqNum=2 2023-02-13 21:59:56,084 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=6bd825938f619ef4084c64b1f66ce2a6, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,46497,1676325587650, regionLocation=jenkins-hbase20.apache.org,46497,1676325587650, openSeqNum=2 2023-02-13 21:59:56,084 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 21:59:56,084 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325656084 2023-02-13 21:59:56,084 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325716084 2023-02-13 21:59:56,084 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 9 msec 2023-02-13 21:59:56,104 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,46045,1676325594602-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:56,104 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,46045,1676325594602-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:56,104 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,46045,1676325594602-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:56,104 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase20:46045, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:56,104 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:56,104 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase20.apache.org,46497,1676325587650 had 2 regions 2023-02-13 21:59:56,105 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on existing znode=/hbase/namespace 2023-02-13 21:59:56,106 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase20.apache.org,46497,1676325587650, splitWal=true, meta=true, isMeta: false 2023-02-13 21:59:56,108 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,46497,1676325587650-splitting dir is empty, no logs to split. 2023-02-13 21:59:56,109 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase20.apache.org,46497,1676325587650 WAL count=0, meta=false 2023-02-13 21:59:56,110 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 21:59:56,111 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:39320, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=ClientService 2023-02-13 21:59:56,113 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/WALs/jenkins-hbase20.apache.org,46497,1676325587650-splitting dir is empty, no logs to split. 2023-02-13 21:59:56,113 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase20.apache.org,46497,1676325587650 WAL count=0, meta=false 2023-02-13 21:59:56,113 WARN [master/jenkins-hbase20:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase20.apache.org,46497,1676325587650/restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:56,113 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase20.apache.org,46497,1676325587650 WAL splitting is done? wals=0, meta=false 2023-02-13 21:59:56,116 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableTwo, region=6bd825938f619ef4084c64b1f66ce2a6, ASSIGN}] 2023-02-13 21:59:56,117 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableTwo, region=6bd825938f619ef4084c64b1f66ce2a6, ASSIGN 2023-02-13 21:59:56,117 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 21:59:56,119 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(189): Setting lastHost as the region location jenkins-hbase20.apache.org,46497,1676325587650 2023-02-13 21:59:56,119 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=21, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=restartTableTwo, region=6bd825938f619ef4084c64b1f66ce2a6, ASSIGN; state=OPEN, location=jenkins-hbase20.apache.org,46497,1676325587650; forceNewPlan=false, retain=true 2023-02-13 21:59:56,119 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 21:59:56,119 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 1.481sec 2023-02-13 21:59:56,119 INFO [master/jenkins-hbase20:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 21:59:56,120 INFO [master/jenkins-hbase20:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 21:59:56,120 INFO [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 21:59:56,120 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,46045,1676325594602-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 21:59:56,120 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,46045,1676325594602-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 21:59:56,127 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 21:59:56,269 INFO [jenkins-hbase20:46045] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 21:59:56,270 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=6bd825938f619ef4084c64b1f66ce2a6, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:56,270 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325596270"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325596270"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325596270"}]},"ts":"1676325596270"} 2023-02-13 21:59:56,274 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; OpenRegionProcedure 6bd825938f619ef4084c64b1f66ce2a6, server=jenkins-hbase20.apache.org,46497,1676325594651}] 2023-02-13 21:59:56,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:46497-0] handler.AssignRegionHandler(128): Open restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:56,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6bd825938f619ef4084c64b1f66ce2a6, NAME => 'restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6.', STARTKEY => '', ENDKEY => ''} 2023-02-13 21:59:56,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table restartTableTwo 6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:56,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(865): Instantiated restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:56,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(7890): checking encryption for 6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:56,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(7893): checking classloading for 6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:56,434 INFO [StoreOpener-6bd825938f619ef4084c64b1f66ce2a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family of region 6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:56,435 DEBUG [StoreOpener-6bd825938f619ef4084c64b1f66ce2a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableTwo/6bd825938f619ef4084c64b1f66ce2a6/family 2023-02-13 21:59:56,435 DEBUG [StoreOpener-6bd825938f619ef4084c64b1f66ce2a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableTwo/6bd825938f619ef4084c64b1f66ce2a6/family 2023-02-13 21:59:56,436 INFO [StoreOpener-6bd825938f619ef4084c64b1f66ce2a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6bd825938f619ef4084c64b1f66ce2a6 columnFamilyName family 2023-02-13 21:59:56,437 INFO [StoreOpener-6bd825938f619ef4084c64b1f66ce2a6-1] regionserver.HStore(310): Store=6bd825938f619ef4084c64b1f66ce2a6/family, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:56,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableTwo/6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:56,441 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableTwo/6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:56,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1054): writing seq id for 6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:56,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1071): Opened 6bd825938f619ef4084c64b1f66ce2a6; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62327324, jitterRate=-0.07125049829483032}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 21:59:56,445 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(964): Region open journal for 6bd825938f619ef4084c64b1f66ce2a6: Running coprocessor pre-open hook at 1676325596432Writing region info on filesystem at 1676325596432Initializing all the Stores at 1676325596433 (+1 ms)Instantiating store for column family {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325596433Cleaning up temporary data from old regions at 1676325596441 (+8 ms)Cleaning up detritus from prior splits at 1676325596442 (+1 ms)Running coprocessor post-open hooks at 1676325596445 (+3 ms)Region opened successfully at 1676325596445 2023-02-13 21:59:56,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegionServer(2335): Post open deploy tasks for restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6., pid=22, masterSystemTime=1676325596427 2023-02-13 21:59:56,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegionServer(2362): Finished post open deploy task for restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:56,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:46497-0] handler.AssignRegionHandler(156): Opened restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:56,450 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=6bd825938f619ef4084c64b1f66ce2a6, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:56,452 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325596450"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325596450"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325596450"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325596450"}]},"ts":"1676325596450"} 2023-02-13 21:59:56,457 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-13 21:59:56,457 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; OpenRegionProcedure 6bd825938f619ef4084c64b1f66ce2a6, server=jenkins-hbase20.apache.org,46497,1676325594651 in 180 msec 2023-02-13 21:59:56,460 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=18 2023-02-13 21:59:56,461 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase20.apache.org,46497,1676325587650 after splitting done 2023-02-13 21:59:56,461 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=restartTableTwo, region=6bd825938f619ef4084c64b1f66ce2a6, ASSIGN in 341 msec 2023-02-13 21:59:56,461 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase20.apache.org,46497,1676325587650 from processing; numProcessing=0 2023-02-13 21:59:56,463 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase20.apache.org,46497,1676325587650, splitWal=true, meta=true in 1.5910 sec 2023-02-13 21:59:56,526 DEBUG [hconnection-0x2653f680-shared-pool-6] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 21:59:56,528 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:39322, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 21:59:56,531 INFO [Listener at localhost.localdomain/34171] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 21:59:56,543 INFO [Listener at localhost.localdomain/34171] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 21:59:56,552 DEBUG [hconnection-0x2653f680-shared-pool-6] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 21:59:56,553 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:39862, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 21:59:56,556 INFO [Listener at localhost.localdomain/34171] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 21:59:56,556 INFO [Listener at localhost.localdomain/34171] master.SnapshotOfRegionAssignmentFromMeta(102): Start to scan the hbase:meta for the current region assignment snappshot 2023-02-13 21:59:56,560 INFO [Listener at localhost.localdomain/34171] master.SnapshotOfRegionAssignmentFromMeta(164): Finished to scan the hbase:meta for the current region assignmentsnapshot 2023-02-13 21:59:56,561 INFO [Listener at localhost.localdomain/34171] master.TestRetainAssignmentOnRestart(187): Key={ENCODED => 6bd825938f619ef4084c64b1f66ce2a6, NAME => 'restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6.', STARTKEY => '', ENDKEY => ''} oldServer=jenkins-hbase20.apache.org,46497,1676325587650, currentServer=jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:56,561 INFO [Listener at localhost.localdomain/34171] master.TestRetainAssignmentOnRestart(187): Key={ENCODED => 364f7b221bf1134db4d8b7aca4f9c7a0, NAME => 'restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0.', STARTKEY => '', ENDKEY => ''} oldServer=jenkins-hbase20.apache.org,34119,1676325587690, currentServer=jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:56,561 INFO [Listener at localhost.localdomain/34171] master.TestRetainAssignmentOnRestart(187): Key={ENCODED => 6a1ed9b3fbb3a5a337d9be9c45e6e831, NAME => 'restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831.', STARTKEY => '', ENDKEY => ''} oldServer=jenkins-hbase20.apache.org,45487,1676325587722, currentServer=jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:56,561 INFO [Listener at localhost.localdomain/34171] master.TestRetainAssignmentOnRestart(187): Key={ENCODED => b2169e2475351a310eb3323519b34622, NAME => 'hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622.', STARTKEY => '', ENDKEY => ''} oldServer=jenkins-hbase20.apache.org,34119,1676325587690, currentServer=jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:56,561 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-13 21:59:56,561 INFO [Listener at localhost.localdomain/34171] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 21:59:56,561 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6263f8ce to 127.0.0.1:57335 2023-02-13 21:59:56,562 DEBUG [Listener at localhost.localdomain/34171] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 21:59:56,562 DEBUG [Listener at localhost.localdomain/34171] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-13 21:59:56,562 DEBUG [Listener at localhost.localdomain/34171] util.JVMClusterUtil(257): Found active master hash=1652182553, stopped=false 2023-02-13 21:59:56,562 INFO [Listener at localhost.localdomain/34171] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase20.apache.org,46045,1676325594602 2023-02-13 21:59:56,564 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:46497-0x1015dda4573000a, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 21:59:56,564 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 21:59:56,564 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 21:59:56,564 INFO [Listener at localhost.localdomain/34171] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 21:59:56,564 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 21:59:56,564 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:56,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 21:59:56,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 21:59:56,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46497-0x1015dda4573000a, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 21:59:56,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 21:59:56,566 DEBUG [Listener at localhost.localdomain/34171] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3e78901a to 127.0.0.1:57335 2023-02-13 21:59:56,566 DEBUG [Listener at localhost.localdomain/34171] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 21:59:56,566 INFO [Listener at localhost.localdomain/34171] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,34119,1676325587690' ***** 2023-02-13 21:59:56,566 INFO [Listener at localhost.localdomain/34171] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 21:59:56,566 INFO [Listener at localhost.localdomain/34171] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,45487,1676325587722' ***** 2023-02-13 21:59:56,566 INFO [Listener at localhost.localdomain/34171] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 21:59:56,567 INFO [Listener at localhost.localdomain/34171] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,46497,1676325594651' ***** 2023-02-13 21:59:56,567 INFO [Listener at localhost.localdomain/34171] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 21:59:56,567 INFO [RS:1;jenkins-hbase20:34119] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 21:59:56,567 INFO [RS:3;jenkins-hbase20:46497] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 21:59:56,567 INFO [RS:1;jenkins-hbase20:34119] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 21:59:56,567 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 21:59:56,567 INFO [RS:1;jenkins-hbase20:34119] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 21:59:56,567 INFO [RS:3;jenkins-hbase20:46497] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 21:59:56,567 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 21:59:56,567 INFO [RS:2;jenkins-hbase20:45487] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 21:59:56,568 INFO [RS:3;jenkins-hbase20:46497] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 21:59:56,568 INFO [RS:2;jenkins-hbase20:45487] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 21:59:56,568 INFO [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(3304): Received CLOSE for 6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:56,569 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 21:59:56,568 INFO [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(3304): Received CLOSE for b2169e2475351a310eb3323519b34622 2023-02-13 21:59:56,569 INFO [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:56,568 INFO [RS:2;jenkins-hbase20:45487] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 21:59:56,569 DEBUG [RS:3;jenkins-hbase20:46497] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0393332b to 127.0.0.1:57335 2023-02-13 21:59:56,570 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1603): Closing 6bd825938f619ef4084c64b1f66ce2a6, disabling compactions & flushes 2023-02-13 21:59:56,570 DEBUG [RS:3;jenkins-hbase20:46497] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 21:59:56,570 INFO [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(3304): Received CLOSE for 364f7b221bf1134db4d8b7aca4f9c7a0 2023-02-13 21:59:56,569 INFO [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(3304): Received CLOSE for 6a1ed9b3fbb3a5a337d9be9c45e6e831 2023-02-13 21:59:56,571 INFO [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:56,571 DEBUG [RS:1;jenkins-hbase20:34119] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x28ea3142 to 127.0.0.1:57335 2023-02-13 21:59:56,571 DEBUG [RS:1;jenkins-hbase20:34119] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 21:59:56,570 INFO [RS:3;jenkins-hbase20:46497] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 21:59:56,571 INFO [RS:3;jenkins-hbase20:46497] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 21:59:56,571 INFO [RS:3;jenkins-hbase20:46497] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 21:59:56,570 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing b2169e2475351a310eb3323519b34622, disabling compactions & flushes 2023-02-13 21:59:56,570 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1625): Closing region restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:56,571 INFO [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 21:59:56,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:56,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1713): Acquired close lock on restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. after waiting 0 ms 2023-02-13 21:59:56,572 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622. 2023-02-13 21:59:56,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622. 2023-02-13 21:59:56,572 INFO [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-02-13 21:59:56,572 DEBUG [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(1479): Online Regions={6bd825938f619ef4084c64b1f66ce2a6=restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6., 1588230740=hbase:meta,,1.1588230740} 2023-02-13 21:59:56,572 DEBUG [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(1505): Waiting on 1588230740, 6bd825938f619ef4084c64b1f66ce2a6 2023-02-13 21:59:56,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 6a1ed9b3fbb3a5a337d9be9c45e6e831, disabling compactions & flushes 2023-02-13 21:59:56,571 INFO [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-02-13 21:59:56,571 INFO [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:56,573 DEBUG [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(1479): Online Regions={b2169e2475351a310eb3323519b34622=hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622., 364f7b221bf1134db4d8b7aca4f9c7a0=restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0.} 2023-02-13 21:59:56,573 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831. 2023-02-13 21:59:56,573 DEBUG [RS:2;jenkins-hbase20:45487] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x20e16025 to 127.0.0.1:57335 2023-02-13 21:59:56,573 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 21:59:56,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622. after waiting 0 ms 2023-02-13 21:59:56,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1723): Updates disabled for region restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:56,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622. 2023-02-13 21:59:56,574 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing b2169e2475351a310eb3323519b34622 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-13 21:59:56,573 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 21:59:56,573 DEBUG [RS:2;jenkins-hbase20:45487] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 21:59:56,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831. 2023-02-13 21:59:56,573 DEBUG [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(1505): Waiting on 364f7b221bf1134db4d8b7aca4f9c7a0, b2169e2475351a310eb3323519b34622 2023-02-13 21:59:56,574 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831. after waiting 0 ms 2023-02-13 21:59:56,574 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831. 2023-02-13 21:59:56,574 INFO [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 21:59:56,574 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 21:59:56,575 DEBUG [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(1479): Online Regions={6a1ed9b3fbb3a5a337d9be9c45e6e831=restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831.} 2023-02-13 21:59:56,575 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 21:59:56,575 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 21:59:56,575 DEBUG [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(1505): Waiting on 6a1ed9b3fbb3a5a337d9be9c45e6e831 2023-02-13 21:59:56,575 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=976 B heapSize=2.29 KB 2023-02-13 21:59:56,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:46497-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableTwo/6bd825938f619ef4084c64b1f66ce2a6/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 21:59:56,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableThree/6a1ed9b3fbb3a5a337d9be9c45e6e831/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 21:59:56,611 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1837): Closed restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:56,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1557): Region close journal for 6bd825938f619ef4084c64b1f66ce2a6: Waiting for close lock at 1676325596569Running coprocessor pre-close hooks at 1676325596569Disabling compacts and flushes for region at 1676325596569Disabling writes for close at 1676325596572 (+3 ms)Writing region close event to WAL at 1676325596585 (+13 ms)Running coprocessor post-close hooks at 1676325596610 (+25 ms)Closed at 1676325596611 (+1 ms) 2023-02-13 21:59:56,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:46497-0] handler.CloseRegionHandler(117): Closed restartTableTwo,,1676325591066.6bd825938f619ef4084c64b1f66ce2a6. 2023-02-13 21:59:56,616 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831. 2023-02-13 21:59:56,617 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 6a1ed9b3fbb3a5a337d9be9c45e6e831: Waiting for close lock at 1676325596571Running coprocessor pre-close hooks at 1676325596571Disabling compacts and flushes for region at 1676325596571Disabling writes for close at 1676325596574 (+3 ms)Writing region close event to WAL at 1676325596586 (+12 ms)Running coprocessor post-close hooks at 1676325596610 (+24 ms)Closed at 1676325596616 (+6 ms) 2023-02-13 21:59:56,620 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed restartTableThree,,1676325591684.6a1ed9b3fbb3a5a337d9be9c45e6e831. 2023-02-13 21:59:56,641 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:46497-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=976 B at sequenceid=32 (bloomFilter=false), to=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/.tmp/info/886fd52465be49f5b6514891d6ee6881 2023-02-13 21:59:56,641 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/namespace/b2169e2475351a310eb3323519b34622/.tmp/info/4b06f6688b124ea6b220fc9da9755300 2023-02-13 21:59:56,648 INFO [regionserver/jenkins-hbase20:46497.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 21:59:56,652 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/.tmp/info/886fd52465be49f5b6514891d6ee6881 as hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/info/886fd52465be49f5b6514891d6ee6881 2023-02-13 21:59:56,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/namespace/b2169e2475351a310eb3323519b34622/.tmp/info/4b06f6688b124ea6b220fc9da9755300 as hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/namespace/b2169e2475351a310eb3323519b34622/info/4b06f6688b124ea6b220fc9da9755300 2023-02-13 21:59:56,661 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/namespace/b2169e2475351a310eb3323519b34622/info/4b06f6688b124ea6b220fc9da9755300, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 21:59:56,661 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:46497-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/info/886fd52465be49f5b6514891d6ee6881, entries=8, sequenceid=32, filesize=5.6 K 2023-02-13 21:59:56,664 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for b2169e2475351a310eb3323519b34622 in 90ms, sequenceid=6, compaction requested=false 2023-02-13 21:59:56,664 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(2947): Finished flush of dataSize ~976 B/976, heapSize ~1.77 KB/1816, currentSize=0 B/0 for 1588230740 in 89ms, sequenceid=32, compaction requested=false 2023-02-13 21:59:56,666 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 21:59:56,666 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 21:59:56,685 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:46497-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/meta/1588230740/recovered.edits/35.seqid, newMaxSeqId=35, maxSeqId=27 2023-02-13 21:59:56,685 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/hbase/namespace/b2169e2475351a310eb3323519b34622/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-13 21:59:56,686 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:46497-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 21:59:56,687 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 21:59:56,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622. 2023-02-13 21:59:56,687 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:46497-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676325596572Running coprocessor pre-close hooks at 1676325596572Disabling compacts and flushes for region at 1676325596572Disabling writes for close at 1676325596575 (+3 ms)Obtaining lock to block concurrent updates at 1676325596575Preparing flush snapshotting stores in 1588230740 at 1676325596575Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=976, getHeapSize=2296, getOffHeapSize=0, getCellsCount=8 at 1676325596575Flushing stores of hbase:meta,,1.1588230740 at 1676325596579 (+4 ms)Flushing 1588230740/info: creating writer at 1676325596579Flushing 1588230740/info: appending metadata at 1676325596583 (+4 ms)Flushing 1588230740/info: closing flushed file at 1676325596584 (+1 ms)Flushing 1588230740/info: reopening flushed file at 1676325596653 (+69 ms)Finished flush of dataSize ~976 B/976, heapSize ~1.77 KB/1816, currentSize=0 B/0 for 1588230740 in 89ms, sequenceid=32, compaction requested=false at 1676325596666 (+13 ms)Writing region close event to WAL at 1676325596678 (+12 ms)Running coprocessor post-close hooks at 1676325596686 (+8 ms)Closed at 1676325596687 (+1 ms) 2023-02-13 21:59:56,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for b2169e2475351a310eb3323519b34622: Waiting for close lock at 1676325596570Running coprocessor pre-close hooks at 1676325596570Disabling compacts and flushes for region at 1676325596570Disabling writes for close at 1676325596573 (+3 ms)Obtaining lock to block concurrent updates at 1676325596574 (+1 ms)Preparing flush snapshotting stores in b2169e2475351a310eb3323519b34622 at 1676325596574Finished memstore snapshotting hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676325596574Flushing stores of hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622. at 1676325596579 (+5 ms)Flushing b2169e2475351a310eb3323519b34622/info: creating writer at 1676325596579Flushing b2169e2475351a310eb3323519b34622/info: appending metadata at 1676325596586 (+7 ms)Flushing b2169e2475351a310eb3323519b34622/info: closing flushed file at 1676325596586Flushing b2169e2475351a310eb3323519b34622/info: reopening flushed file at 1676325596653 (+67 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for b2169e2475351a310eb3323519b34622 in 90ms, sequenceid=6, compaction requested=false at 1676325596664 (+11 ms)Writing region close event to WAL at 1676325596675 (+11 ms)Running coprocessor post-close hooks at 1676325596686 (+11 ms)Closed at 1676325596687 (+1 ms) 2023-02-13 21:59:56,687 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:46497-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 21:59:56,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676325589624.b2169e2475351a310eb3323519b34622. 2023-02-13 21:59:56,688 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 364f7b221bf1134db4d8b7aca4f9c7a0, disabling compactions & flushes 2023-02-13 21:59:56,688 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0. 2023-02-13 21:59:56,688 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0. 2023-02-13 21:59:56,688 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0. after waiting 0 ms 2023-02-13 21:59:56,688 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0. 2023-02-13 21:59:56,694 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/data/default/restartTableOne/364f7b221bf1134db4d8b7aca4f9c7a0/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 21:59:56,695 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0. 2023-02-13 21:59:56,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 364f7b221bf1134db4d8b7aca4f9c7a0: Waiting for close lock at 1676325596687Running coprocessor pre-close hooks at 1676325596687Disabling compacts and flushes for region at 1676325596687Disabling writes for close at 1676325596688 (+1 ms)Writing region close event to WAL at 1676325596689 (+1 ms)Running coprocessor post-close hooks at 1676325596695 (+6 ms)Closed at 1676325596695 2023-02-13 21:59:56,696 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed restartTableOne,,1676325590423.364f7b221bf1134db4d8b7aca4f9c7a0. 2023-02-13 21:59:56,772 INFO [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,46497,1676325594651; all regions closed. 2023-02-13 21:59:56,775 INFO [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,34119,1676325587690; all regions closed. 2023-02-13 21:59:56,775 INFO [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,45487,1676325587722; all regions closed. 2023-02-13 21:59:56,783 DEBUG [RS:3;jenkins-hbase20:46497] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/oldWALs 2023-02-13 21:59:56,783 INFO [RS:3;jenkins-hbase20:46497] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C46497%2C1676325594651.meta:.meta(num 1676325595260) 2023-02-13 21:59:56,789 DEBUG [RS:1;jenkins-hbase20:34119] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/oldWALs 2023-02-13 21:59:56,789 INFO [RS:1;jenkins-hbase20:34119] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C34119%2C1676325587690:(num 1676325589217) 2023-02-13 21:59:56,789 DEBUG [RS:1;jenkins-hbase20:34119] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 21:59:56,789 DEBUG [RS:2;jenkins-hbase20:45487] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/oldWALs 2023-02-13 21:59:56,789 INFO [RS:1;jenkins-hbase20:34119] regionserver.LeaseManager(133): Closed leases 2023-02-13 21:59:56,789 INFO [RS:2;jenkins-hbase20:45487] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C45487%2C1676325587722:(num 1676325589217) 2023-02-13 21:59:56,790 DEBUG [RS:2;jenkins-hbase20:45487] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 21:59:56,790 INFO [RS:2;jenkins-hbase20:45487] regionserver.LeaseManager(133): Closed leases 2023-02-13 21:59:56,790 INFO [RS:1;jenkins-hbase20:34119] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 21:59:56,790 INFO [RS:1;jenkins-hbase20:34119] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 21:59:56,790 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 21:59:56,790 INFO [RS:2;jenkins-hbase20:45487] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 21:59:56,790 INFO [RS:2;jenkins-hbase20:45487] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 21:59:56,790 INFO [RS:2;jenkins-hbase20:45487] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 21:59:56,790 INFO [RS:2;jenkins-hbase20:45487] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 21:59:56,790 INFO [RS:1;jenkins-hbase20:34119] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 21:59:56,791 INFO [RS:2;jenkins-hbase20:45487] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:45487 2023-02-13 21:59:56,790 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 21:59:56,791 INFO [RS:1;jenkins-hbase20:34119] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 21:59:56,793 INFO [RS:1;jenkins-hbase20:34119] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:34119 2023-02-13 21:59:56,794 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:46497-0x1015dda4573000a, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:56,795 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 21:59:56,795 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:56,795 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@20edbddd rejected from java.util.concurrent.ThreadPoolExecutor@24d22d18[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:56,794 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,45487,1676325587722 2023-02-13 21:59:56,796 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 21:59:56,795 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 21:59:56,796 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1db5b4b2 rejected from java.util.concurrent.ThreadPoolExecutor@24d22d18[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:56,795 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:46497-0x1015dda4573000a, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 21:59:56,796 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:46497-0x1015dda4573000a, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:56,796 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,45487,1676325587722] 2023-02-13 21:59:56,796 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,45487,1676325587722; numProcessing=1 2023-02-13 21:59:56,797 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:56,796 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,34119,1676325587690 2023-02-13 21:59:56,797 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@65f4ac76 rejected from java.util.concurrent.ThreadPoolExecutor@24d22d18[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:56,798 DEBUG [RS:3;jenkins-hbase20:46497] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/oldWALs 2023-02-13 21:59:56,798 INFO [RS:3;jenkins-hbase20:46497] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C46497%2C1676325594651:(num 1676325595176) 2023-02-13 21:59:56,798 DEBUG [RS:3;jenkins-hbase20:46497] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 21:59:56,799 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,45487,1676325587722 already deleted, retry=false 2023-02-13 21:59:56,799 INFO [RS:3;jenkins-hbase20:46497] regionserver.LeaseManager(133): Closed leases 2023-02-13 21:59:56,799 INFO [RS:3;jenkins-hbase20:46497] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:46497 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 21:59:56,799 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,45487,1676325587722 expired; onlineServers=2 2023-02-13 21:59:56,799 INFO [regionserver/jenkins-hbase20:46497.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 21:59:56,799 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,34119,1676325587690] 2023-02-13 21:59:56,799 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,34119,1676325587690; numProcessing=2 2023-02-13 21:59:56,800 INFO [RS:3;jenkins-hbase20:46497] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:46497 2023-02-13 21:59:56,802 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 21:59:56,803 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,34119,1676325587690 already deleted, retry=false 2023-02-13 21:59:56,802 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:46497-0x1015dda4573000a, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,46497,1676325594651 2023-02-13 21:59:56,803 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,34119,1676325587690 expired; onlineServers=1 2023-02-13 21:59:56,803 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7b860365 rejected from java.util.concurrent.ThreadPoolExecutor@7fb988cd[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 5] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:56,804 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,46497,1676325594651] 2023-02-13 21:59:56,804 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,46497,1676325594651; numProcessing=3 2023-02-13 21:59:56,804 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,46497,1676325594651 already deleted, retry=false 2023-02-13 21:59:56,804 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,46497,1676325594651 expired; onlineServers=0 2023-02-13 21:59:56,804 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,46045,1676325594602' ***** 2023-02-13 21:59:56,804 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-13 21:59:56,805 DEBUG [M:0;jenkins-hbase20:46045] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6645d68e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 21:59:56,805 INFO [M:0;jenkins-hbase20:46045] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,46045,1676325594602 2023-02-13 21:59:56,805 INFO [M:0;jenkins-hbase20:46045] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,46045,1676325594602; all regions closed. 2023-02-13 21:59:56,805 DEBUG [M:0;jenkins-hbase20:46045] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 21:59:56,806 DEBUG [M:0;jenkins-hbase20:46045] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 21:59:56,806 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 21:59:56,806 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676325594921] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676325594921,5,FailOnTimeoutGroup] 2023-02-13 21:59:56,806 DEBUG [M:0;jenkins-hbase20:46045] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 21:59:56,806 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676325594922] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676325594922,5,FailOnTimeoutGroup] 2023-02-13 21:59:56,806 INFO [M:0;jenkins-hbase20:46045] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 21:59:56,807 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:56,807 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:56,807 INFO [M:0;jenkins-hbase20:46045] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 21:59:56,807 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 21:59:56,807 INFO [M:0;jenkins-hbase20:46045] hbase.ChoreService(369): Chore service for: master/jenkins-hbase20:0 had [] on shutdown 2023-02-13 21:59:56,808 DEBUG [M:0;jenkins-hbase20:46045] master.HMaster(1502): Stopping service threads 2023-02-13 21:59:56,808 INFO [M:0;jenkins-hbase20:46045] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 21:59:56,808 ERROR [M:0;jenkins-hbase20:46045] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 21:59:56,808 INFO [M:0;jenkins-hbase20:46045] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 21:59:56,808 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 21:59:56,808 DEBUG [M:0;jenkins-hbase20:46045] zookeeper.ZKUtil(398): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 21:59:56,809 WARN [M:0;jenkins-hbase20:46045] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 21:59:56,809 INFO [M:0;jenkins-hbase20:46045] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 21:59:56,809 INFO [M:0;jenkins-hbase20:46045] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 21:59:56,809 DEBUG [M:0;jenkins-hbase20:46045] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 21:59:56,809 INFO [M:0;jenkins-hbase20:46045] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:56,809 DEBUG [M:0;jenkins-hbase20:46045] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:56,809 DEBUG [M:0;jenkins-hbase20:46045] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 21:59:56,810 DEBUG [M:0;jenkins-hbase20:46045] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:56,810 INFO [M:0;jenkins-hbase20:46045] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=15.49 KB heapSize=19.37 KB 2023-02-13 21:59:56,827 INFO [M:0;jenkins-hbase20:46045] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=15.49 KB at sequenceid=186 (bloomFilter=true), to=hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/acd438a34f2448efaf4b35de0be0b9a5 2023-02-13 21:59:56,834 DEBUG [M:0;jenkins-hbase20:46045] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/acd438a34f2448efaf4b35de0be0b9a5 as hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/acd438a34f2448efaf4b35de0be0b9a5 2023-02-13 21:59:56,842 INFO [M:0;jenkins-hbase20:46045] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36521/user/jenkins/test-data/bb3dfbfe-09e5-2175-121e-0ef9b271192f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/acd438a34f2448efaf4b35de0be0b9a5, entries=5, sequenceid=186, filesize=5.3 K 2023-02-13 21:59:56,843 INFO [M:0;jenkins-hbase20:46045] regionserver.HRegion(2947): Finished flush of dataSize ~15.49 KB/15863, heapSize ~19.35 KB/19816, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 33ms, sequenceid=186, compaction requested=false 2023-02-13 21:59:56,845 INFO [M:0;jenkins-hbase20:46045] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:56,845 DEBUG [M:0;jenkins-hbase20:46045] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676325596809Disabling compacts and flushes for region at 1676325596809Disabling writes for close at 1676325596810 (+1 ms)Obtaining lock to block concurrent updates at 1676325596810Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676325596810Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=15863, getHeapSize=19816, getOffHeapSize=0, getCellsCount=49 at 1676325596810Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676325596811 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676325596812 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676325596815 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676325596815Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676325596835 (+20 ms)Finished flush of dataSize ~15.49 KB/15863, heapSize ~19.35 KB/19816, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 33ms, sequenceid=186, compaction requested=false at 1676325596843 (+8 ms)Writing region close event to WAL at 1676325596845 (+2 ms)Closed at 1676325596845 2023-02-13 21:59:56,851 INFO [M:0;jenkins-hbase20:46045] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 21:59:56,851 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 21:59:56,851 INFO [M:0;jenkins-hbase20:46045] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:46045 2023-02-13 21:59:56,853 DEBUG [M:0;jenkins-hbase20:46045] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase20.apache.org,46045,1676325594602 already deleted, retry=false 2023-02-13 21:59:57,067 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 21:59:57,067 INFO [M:0;jenkins-hbase20:46045] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,46045,1676325594602; zookeeper connection closed. 2023-02-13 21:59:57,067 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2c6062b9 rejected from java.util.concurrent.ThreadPoolExecutor@285fe3ed[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 17] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:57,067 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): master:46045-0x1015dda45730009, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 21:59:57,068 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7320e6d4 rejected from java.util.concurrent.ThreadPoolExecutor@285fe3ed[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 17] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:57,167 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:46497-0x1015dda4573000a, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 21:59:57,167 INFO [RS:3;jenkins-hbase20:46497] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,46497,1676325594651; zookeeper connection closed. 2023-02-13 21:59:57,168 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@401d59bf rejected from java.util.concurrent.ThreadPoolExecutor@7fb988cd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 5] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:57,168 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:46497-0x1015dda4573000a, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 21:59:57,168 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@317520b4] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@317520b4 2023-02-13 21:59:57,168 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3887f9a6 rejected from java.util.concurrent.ThreadPoolExecutor@7fb988cd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 5] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:57,268 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 21:59:57,268 INFO [RS:1;jenkins-hbase20:34119] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,34119,1676325587690; zookeeper connection closed. 2023-02-13 21:59:57,268 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3e20e19c rejected from java.util.concurrent.ThreadPoolExecutor@3a1f6280[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 12] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:57,268 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6319cbec] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6319cbec 2023-02-13 21:59:57,268 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:34119-0x1015dda45730002, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 21:59:57,268 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@773dc53d rejected from java.util.concurrent.ThreadPoolExecutor@3a1f6280[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 12] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:57,368 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 21:59:57,368 INFO [RS:2;jenkins-hbase20:45487] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,45487,1676325587722; zookeeper connection closed. 2023-02-13 21:59:57,369 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@31df7c00 rejected from java.util.concurrent.ThreadPoolExecutor@24d22d18[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:57,369 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4da1731b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4da1731b 2023-02-13 21:59:57,369 DEBUG [Listener at localhost.localdomain/34171-EventThread] zookeeper.ZKWatcher(600): regionserver:45487-0x1015dda45730003, quorum=127.0.0.1:57335, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 21:59:57,369 ERROR [Listener at localhost.localdomain/34171-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7ef08098 rejected from java.util.concurrent.ThreadPoolExecutor@24d22d18[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 21:59:57,369 INFO [Listener at localhost.localdomain/34171] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 4 regionserver(s) complete 2023-02-13 21:59:57,370 WARN [Listener at localhost.localdomain/34171] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 21:59:57,372 INFO [Listener at localhost.localdomain/34171] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 21:59:57,448 WARN [BP-696549777-148.251.75.209-1676325583640 heartbeating to localhost.localdomain/127.0.0.1:36521] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-696549777-148.251.75.209-1676325583640 (Datanode Uuid 8e39e35d-a56a-4b9c-a217-5e2392a9f1a9) service to localhost.localdomain/127.0.0.1:36521 2023-02-13 21:59:57,451 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/cluster_0697ba01-9152-a749-74f7-a697351678b4/dfs/data/data5/current/BP-696549777-148.251.75.209-1676325583640] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 21:59:57,451 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/cluster_0697ba01-9152-a749-74f7-a697351678b4/dfs/data/data6/current/BP-696549777-148.251.75.209-1676325583640] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 21:59:57,478 WARN [Listener at localhost.localdomain/34171] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 21:59:57,480 INFO [Listener at localhost.localdomain/34171] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 21:59:57,583 WARN [BP-696549777-148.251.75.209-1676325583640 heartbeating to localhost.localdomain/127.0.0.1:36521] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 21:59:57,583 WARN [BP-696549777-148.251.75.209-1676325583640 heartbeating to localhost.localdomain/127.0.0.1:36521] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-696549777-148.251.75.209-1676325583640 (Datanode Uuid d8f56787-7e58-4399-ae6d-6fb700e1d7bd) service to localhost.localdomain/127.0.0.1:36521 2023-02-13 21:59:57,584 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/cluster_0697ba01-9152-a749-74f7-a697351678b4/dfs/data/data3/current/BP-696549777-148.251.75.209-1676325583640] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 21:59:57,584 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/cluster_0697ba01-9152-a749-74f7-a697351678b4/dfs/data/data4/current/BP-696549777-148.251.75.209-1676325583640] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 21:59:57,585 WARN [Listener at localhost.localdomain/34171] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 21:59:57,587 INFO [Listener at localhost.localdomain/34171] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 21:59:57,689 WARN [BP-696549777-148.251.75.209-1676325583640 heartbeating to localhost.localdomain/127.0.0.1:36521] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 21:59:57,690 WARN [BP-696549777-148.251.75.209-1676325583640 heartbeating to localhost.localdomain/127.0.0.1:36521] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-696549777-148.251.75.209-1676325583640 (Datanode Uuid fdce1170-0de6-4c94-80d6-58e0a1ca2df6) service to localhost.localdomain/127.0.0.1:36521 2023-02-13 21:59:57,690 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/cluster_0697ba01-9152-a749-74f7-a697351678b4/dfs/data/data1/current/BP-696549777-148.251.75.209-1676325583640] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 21:59:57,690 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/cluster_0697ba01-9152-a749-74f7-a697351678b4/dfs/data/data2/current/BP-696549777-148.251.75.209-1676325583640] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 21:59:57,717 INFO [Listener at localhost.localdomain/34171] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-02-13 21:59:57,831 INFO [Listener at localhost.localdomain/34171] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-13 21:59:57,877 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-02-13 21:59:57,890 INFO [Listener at localhost.localdomain/34171] hbase.ResourceChecker(175): after: master.TestRetainAssignmentOnRestartSplitWithoutZk#testRetainAssignmentOnSingleRSRestart Thread=103 (was 7) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase20:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase20:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:36521 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: process reaper sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RS-EventLoopGroup-8-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@6db2e019 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:300) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost.localdomain:36521 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:36521 from jenkins.hfs.4 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:36521 from jenkins.hfs.3 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.1@localhost.localdomain:36521 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-5-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase20:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: ReplicationExecutor-0 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager$NodeFailoverWorker.run(ReplicationSourceManager.java:703) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:36521 from jenkins.hfs.0 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: process reaper sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReplicationExecutor-0 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager$NodeFailoverWorker.run(ReplicationSourceManager.java:703) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: process reaper sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:36521 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: regionserver/jenkins-hbase20:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:36521 from jenkins.hfs.1 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost.localdomain:36521 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost.localdomain/34171 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.2@localhost.localdomain:36521 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase20:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase20:46497.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: ForkJoinPool-2-worker-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RS-EventLoopGroup-5-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost.localdomain:36521 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-5-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: process reaper sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: process reaper sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost.localdomain:36521 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:36521 from jenkins.hfs.2 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) - Thread LEAK? -, OpenFileDescriptor=525 (was 260) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=261 (was 208) - SystemLoadAverage LEAK? -, ProcessCount=164 (was 164), AvailableMemoryMB=7190 (was 7822) 2023-02-13 21:59:57,900 INFO [Listener at localhost.localdomain/34171] hbase.ResourceChecker(147): before: master.TestRetainAssignmentOnRestartSplitWithoutZk#testRetainAssignmentOnClusterRestart Thread=103, OpenFileDescriptor=525, MaxFileDescriptor=60000, SystemLoadAverage=261, ProcessCount=164, AvailableMemoryMB=7189 2023-02-13 21:59:57,910 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/hadoop.log.dir so I do NOT create it in target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29 2023-02-13 21:59:57,910 WARN [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(449): hadoop.log.dir property value differs in configuration and system: Configuration=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/../logs while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/hadoop.log.dir Erasing configuration value by system value. 2023-02-13 21:59:57,910 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/hadoop.tmp.dir so I do NOT create it in target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29 2023-02-13 21:59:57,910 WARN [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(449): hadoop.tmp.dir property value differs in configuration and system: Configuration=/tmp/hadoop-jenkins while System=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/20a69ae4-f40b-f362-28de-1d411c95416c/hadoop.tmp.dir Erasing configuration value by system value. 2023-02-13 21:59:57,910 DEBUG [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29 2023-02-13 21:59:57,910 INFO [Listener at localhost.localdomain/34171] master.AbstractTestRestartCluster(45): WAL splitting coordinated by zk false 2023-02-13 21:59:57,910 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-13 21:59:57,910 INFO [Listener at localhost.localdomain/34171] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/cluster_42a35679-fe38-66b4-c507-4e77d8912d93, deleteOnExit=true 2023-02-13 21:59:57,910 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-13 21:59:57,911 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/test.cache.data in system properties and HBase conf 2023-02-13 21:59:57,911 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/hadoop.tmp.dir in system properties and HBase conf 2023-02-13 21:59:57,911 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/hadoop.log.dir in system properties and HBase conf 2023-02-13 21:59:57,911 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-13 21:59:57,911 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-13 21:59:57,911 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-13 21:59:57,911 DEBUG [Listener at localhost.localdomain/34171] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-13 21:59:57,912 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-13 21:59:57,912 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-13 21:59:57,912 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-13 21:59:57,912 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 21:59:57,912 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-13 21:59:57,912 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-13 21:59:57,912 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 21:59:57,912 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 21:59:57,913 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-13 21:59:57,913 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/nfs.dump.dir in system properties and HBase conf 2023-02-13 21:59:57,913 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/java.io.tmpdir in system properties and HBase conf 2023-02-13 21:59:57,913 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 21:59:57,913 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-13 21:59:57,913 INFO [Listener at localhost.localdomain/34171] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-13 21:59:57,916 WARN [Listener at localhost.localdomain/34171] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 21:59:57,917 WARN [Listener at localhost.localdomain/34171] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 21:59:57,943 WARN [Listener at localhost.localdomain/34171] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 21:59:57,945 INFO [Listener at localhost.localdomain/34171] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 21:59:57,950 INFO [Listener at localhost.localdomain/34171] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/java.io.tmpdir/Jetty_localhost_localdomain_33725_hdfs____.mg94gc/webapp 2023-02-13 21:59:58,022 INFO [Listener at localhost.localdomain/34171] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:33725 2023-02-13 21:59:58,025 WARN [Listener at localhost.localdomain/34171] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 21:59:58,026 WARN [Listener at localhost.localdomain/34171] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 21:59:58,058 WARN [Listener at localhost.localdomain/34137] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 21:59:58,074 WARN [Listener at localhost.localdomain/34137] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 21:59:58,079 WARN [Listener at localhost.localdomain/34137] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 21:59:58,080 INFO [Listener at localhost.localdomain/34137] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 21:59:58,087 INFO [Listener at localhost.localdomain/34137] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/java.io.tmpdir/Jetty_localhost_43229_datanode____.mpw0sf/webapp 2023-02-13 21:59:58,168 INFO [Listener at localhost.localdomain/34137] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43229 2023-02-13 21:59:58,175 WARN [Listener at localhost.localdomain/37311] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 21:59:58,194 WARN [Listener at localhost.localdomain/37311] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 21:59:58,199 WARN [Listener at localhost.localdomain/37311] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 21:59:58,200 INFO [Listener at localhost.localdomain/37311] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 21:59:58,206 INFO [Listener at localhost.localdomain/37311] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/java.io.tmpdir/Jetty_localhost_46213_datanode____.aknu2v/webapp 2023-02-13 21:59:58,282 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3517ac4456320414: Processing first storage report for DS-3c7651be-1d01-4017-8e7b-be8384d93e79 from datanode 1d43f1a7-51ee-4916-a9b2-7f176ee12168 2023-02-13 21:59:58,283 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3517ac4456320414: from storage DS-3c7651be-1d01-4017-8e7b-be8384d93e79 node DatanodeRegistration(127.0.0.1:37895, datanodeUuid=1d43f1a7-51ee-4916-a9b2-7f176ee12168, infoPort=37795, infoSecurePort=0, ipcPort=37311, storageInfo=lv=-57;cid=testClusterID;nsid=608834342;c=1676325597918), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-13 21:59:58,283 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3517ac4456320414: Processing first storage report for DS-22566d48-eade-4187-a21f-a50d991c7895 from datanode 1d43f1a7-51ee-4916-a9b2-7f176ee12168 2023-02-13 21:59:58,283 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3517ac4456320414: from storage DS-22566d48-eade-4187-a21f-a50d991c7895 node DatanodeRegistration(127.0.0.1:37895, datanodeUuid=1d43f1a7-51ee-4916-a9b2-7f176ee12168, infoPort=37795, infoSecurePort=0, ipcPort=37311, storageInfo=lv=-57;cid=testClusterID;nsid=608834342;c=1676325597918), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 21:59:58,313 INFO [Listener at localhost.localdomain/37311] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46213 2023-02-13 21:59:58,323 WARN [Listener at localhost.localdomain/37941] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 21:59:58,348 WARN [Listener at localhost.localdomain/37941] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 21:59:58,352 WARN [Listener at localhost.localdomain/37941] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 21:59:58,354 INFO [Listener at localhost.localdomain/37941] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 21:59:58,363 INFO [Listener at localhost.localdomain/37941] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/java.io.tmpdir/Jetty_localhost_43353_datanode____68saah/webapp 2023-02-13 21:59:58,430 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfda7de1c6b99015e: Processing first storage report for DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812 from datanode cbf28cde-5b32-4679-879d-8dad84d19144 2023-02-13 21:59:58,430 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfda7de1c6b99015e: from storage DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812 node DatanodeRegistration(127.0.0.1:35895, datanodeUuid=cbf28cde-5b32-4679-879d-8dad84d19144, infoPort=37711, infoSecurePort=0, ipcPort=37941, storageInfo=lv=-57;cid=testClusterID;nsid=608834342;c=1676325597918), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 21:59:58,430 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfda7de1c6b99015e: Processing first storage report for DS-de99fb25-8c9f-435f-ae13-b7bceeaa3db9 from datanode cbf28cde-5b32-4679-879d-8dad84d19144 2023-02-13 21:59:58,430 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfda7de1c6b99015e: from storage DS-de99fb25-8c9f-435f-ae13-b7bceeaa3db9 node DatanodeRegistration(127.0.0.1:35895, datanodeUuid=cbf28cde-5b32-4679-879d-8dad84d19144, infoPort=37711, infoSecurePort=0, ipcPort=37941, storageInfo=lv=-57;cid=testClusterID;nsid=608834342;c=1676325597918), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 21:59:58,459 INFO [Listener at localhost.localdomain/37941] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43353 2023-02-13 21:59:58,472 WARN [Listener at localhost.localdomain/39067] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 21:59:58,569 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa1958a9bf59c4abf: Processing first storage report for DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720 from datanode 0b4e47ea-7dd1-4885-807c-0a6cf357a49f 2023-02-13 21:59:58,569 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa1958a9bf59c4abf: from storage DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720 node DatanodeRegistration(127.0.0.1:37781, datanodeUuid=0b4e47ea-7dd1-4885-807c-0a6cf357a49f, infoPort=40253, infoSecurePort=0, ipcPort=39067, storageInfo=lv=-57;cid=testClusterID;nsid=608834342;c=1676325597918), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 21:59:58,569 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa1958a9bf59c4abf: Processing first storage report for DS-b390d357-b3bb-4a46-a49c-7d53b81580ad from datanode 0b4e47ea-7dd1-4885-807c-0a6cf357a49f 2023-02-13 21:59:58,569 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa1958a9bf59c4abf: from storage DS-b390d357-b3bb-4a46-a49c-7d53b81580ad node DatanodeRegistration(127.0.0.1:37781, datanodeUuid=0b4e47ea-7dd1-4885-807c-0a6cf357a49f, infoPort=40253, infoSecurePort=0, ipcPort=39067, storageInfo=lv=-57;cid=testClusterID;nsid=608834342;c=1676325597918), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 21:59:58,587 DEBUG [Listener at localhost.localdomain/39067] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29 2023-02-13 21:59:58,590 INFO [Listener at localhost.localdomain/39067] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/cluster_42a35679-fe38-66b4-c507-4e77d8912d93/zookeeper_0, clientPort=53523, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/cluster_42a35679-fe38-66b4-c507-4e77d8912d93/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/cluster_42a35679-fe38-66b4-c507-4e77d8912d93/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-13 21:59:58,592 INFO [Listener at localhost.localdomain/39067] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=53523 2023-02-13 21:59:58,593 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:58,594 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:58,617 INFO [Listener at localhost.localdomain/39067] util.FSUtils(479): Created version file at hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2 with version=8 2023-02-13 21:59:58,617 INFO [Listener at localhost.localdomain/39067] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/hbase-staging 2023-02-13 21:59:58,619 INFO [Listener at localhost.localdomain/39067] client.ConnectionUtils(127): master/jenkins-hbase20:0 server-side Connection retries=45 2023-02-13 21:59:58,619 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:58,619 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:58,620 INFO [Listener at localhost.localdomain/39067] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 21:59:58,620 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:58,620 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 21:59:58,620 INFO [Listener at localhost.localdomain/39067] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 21:59:58,620 INFO [Listener at localhost.localdomain/39067] ipc.NettyRpcServer(120): Bind to /148.251.75.209:39807 2023-02-13 21:59:58,621 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:58,622 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:58,623 INFO [Listener at localhost.localdomain/39067] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39807 connecting to ZooKeeper ensemble=127.0.0.1:53523 2023-02-13 21:59:58,630 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:398070x0, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 21:59:58,632 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:39807-0x1015dda772d0000 connected 2023-02-13 21:59:58,648 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 21:59:58,649 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 21:59:58,650 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 21:59:58,651 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39807 2023-02-13 21:59:58,655 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39807 2023-02-13 21:59:58,655 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39807 2023-02-13 21:59:58,658 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39807 2023-02-13 21:59:58,658 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39807 2023-02-13 21:59:58,659 INFO [Listener at localhost.localdomain/39067] master.HMaster(439): hbase.rootdir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2, hbase.cluster.distributed=false 2023-02-13 21:59:58,678 INFO [Listener at localhost.localdomain/39067] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-02-13 21:59:58,678 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:58,678 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:58,679 INFO [Listener at localhost.localdomain/39067] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 21:59:58,679 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:58,679 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 21:59:58,679 INFO [Listener at localhost.localdomain/39067] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 21:59:58,680 INFO [Listener at localhost.localdomain/39067] ipc.NettyRpcServer(120): Bind to /148.251.75.209:37039 2023-02-13 21:59:58,680 INFO [Listener at localhost.localdomain/39067] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 21:59:58,682 DEBUG [Listener at localhost.localdomain/39067] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 21:59:58,683 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:58,684 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:58,686 INFO [Listener at localhost.localdomain/39067] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37039 connecting to ZooKeeper ensemble=127.0.0.1:53523 2023-02-13 21:59:58,690 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:370390x0, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 21:59:58,691 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): regionserver:370390x0, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 21:59:58,691 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:37039-0x1015dda772d0001 connected 2023-02-13 21:59:58,692 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 21:59:58,693 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 21:59:58,699 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37039 2023-02-13 21:59:58,699 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37039 2023-02-13 21:59:58,699 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37039 2023-02-13 21:59:58,703 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37039 2023-02-13 21:59:58,707 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37039 2023-02-13 21:59:58,716 INFO [Listener at localhost.localdomain/39067] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-02-13 21:59:58,717 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:58,717 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:58,717 INFO [Listener at localhost.localdomain/39067] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 21:59:58,717 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:58,717 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 21:59:58,717 INFO [Listener at localhost.localdomain/39067] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 21:59:58,718 INFO [Listener at localhost.localdomain/39067] ipc.NettyRpcServer(120): Bind to /148.251.75.209:45945 2023-02-13 21:59:58,719 INFO [Listener at localhost.localdomain/39067] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 21:59:58,720 DEBUG [Listener at localhost.localdomain/39067] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 21:59:58,720 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:58,721 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:58,722 INFO [Listener at localhost.localdomain/39067] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45945 connecting to ZooKeeper ensemble=127.0.0.1:53523 2023-02-13 21:59:58,726 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:459450x0, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 21:59:58,728 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): regionserver:459450x0, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 21:59:58,729 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): regionserver:459450x0, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 21:59:58,733 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:45945-0x1015dda772d0002 connected 2023-02-13 21:59:58,734 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): regionserver:45945-0x1015dda772d0002, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 21:59:58,735 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45945 2023-02-13 21:59:58,736 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45945 2023-02-13 21:59:58,737 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45945 2023-02-13 21:59:58,739 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45945 2023-02-13 21:59:58,739 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45945 2023-02-13 21:59:58,752 INFO [Listener at localhost.localdomain/39067] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-02-13 21:59:58,752 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:58,752 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:58,752 INFO [Listener at localhost.localdomain/39067] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 21:59:58,752 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 21:59:58,752 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 21:59:58,752 INFO [Listener at localhost.localdomain/39067] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 21:59:58,753 INFO [Listener at localhost.localdomain/39067] ipc.NettyRpcServer(120): Bind to /148.251.75.209:40899 2023-02-13 21:59:58,754 INFO [Listener at localhost.localdomain/39067] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 21:59:58,755 DEBUG [Listener at localhost.localdomain/39067] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 21:59:58,756 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:58,757 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:58,759 INFO [Listener at localhost.localdomain/39067] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40899 connecting to ZooKeeper ensemble=127.0.0.1:53523 2023-02-13 21:59:58,762 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:408990x0, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 21:59:58,764 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): regionserver:408990x0, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 21:59:58,764 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:40899-0x1015dda772d0003 connected 2023-02-13 21:59:58,765 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): regionserver:40899-0x1015dda772d0003, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 21:59:58,765 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): regionserver:40899-0x1015dda772d0003, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 21:59:58,767 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40899 2023-02-13 21:59:58,768 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40899 2023-02-13 21:59:58,771 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40899 2023-02-13 21:59:58,771 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40899 2023-02-13 21:59:58,771 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40899 2023-02-13 21:59:58,773 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase20.apache.org,39807,1676325598619 2023-02-13 21:59:58,774 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 21:59:58,775 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase20.apache.org,39807,1676325598619 2023-02-13 21:59:58,776 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:58,776 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:58,776 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:45945-0x1015dda772d0002, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:58,776 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x1015dda772d0003, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 21:59:58,776 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:58,777 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 21:59:58,778 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase20.apache.org,39807,1676325598619 from backup master directory 2023-02-13 21:59:58,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 21:59:58,779 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase20.apache.org,39807,1676325598619 2023-02-13 21:59:58,779 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 21:59:58,779 WARN [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 21:59:58,779 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase20.apache.org,39807,1676325598619 2023-02-13 21:59:58,806 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/hbase.id with ID: 1de008e8-3020-43bd-a4f6-073a3ec5ca42 2023-02-13 21:59:58,818 INFO [master/jenkins-hbase20:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 21:59:58,820 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:58,836 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x4330f13a to 127.0.0.1:53523 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 21:59:58,839 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7819436f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 21:59:58,839 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 21:59:58,840 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 21:59:58,840 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 21:59:58,842 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store-tmp 2023-02-13 21:59:58,858 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:58,858 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 21:59:58,858 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:58,858 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:58,858 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 21:59:58,858 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:58,858 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 21:59:58,858 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676325598858Disabling compacts and flushes for region at 1676325598858Disabling writes for close at 1676325598858Writing region close event to WAL at 1676325598858Closed at 1676325598858 2023-02-13 21:59:58,859 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/WALs/jenkins-hbase20.apache.org,39807,1676325598619 2023-02-13 21:59:58,862 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C39807%2C1676325598619, suffix=, logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/WALs/jenkins-hbase20.apache.org,39807,1676325598619, archiveDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/oldWALs, maxLogs=10 2023-02-13 21:59:58,878 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK] 2023-02-13 21:59:58,878 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK] 2023-02-13 21:59:58,881 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK] 2023-02-13 21:59:58,884 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/WALs/jenkins-hbase20.apache.org,39807,1676325598619/jenkins-hbase20.apache.org%2C39807%2C1676325598619.1676325598863 2023-02-13 21:59:58,884 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK], DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK], DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK]] 2023-02-13 21:59:58,884 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 21:59:58,885 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:58,885 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:58,885 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:58,888 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:58,890 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 21:59:58,890 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 21:59:58,891 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:58,892 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:58,892 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:58,894 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 21:59:58,897 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 21:59:58,897 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61325666, jitterRate=-0.08617636561393738}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 21:59:58,897 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676325598885Initializing all the Stores at 1676325598886 (+1 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325598886Cleaning up temporary data from old regions at 1676325598892 (+6 ms)Cleaning up detritus from prior splits at 1676325598893 (+1 ms)Region opened successfully at 1676325598897 (+4 ms) 2023-02-13 21:59:58,897 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 21:59:58,899 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-13 21:59:58,899 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-13 21:59:58,899 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 21:59:58,900 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 21:59:58,900 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-02-13 21:59:58,900 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 21:59:58,901 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-13 21:59:58,902 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-13 21:59:58,910 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 21:59:58,910 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 21:59:58,911 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-13 21:59:58,911 INFO [master/jenkins-hbase20:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 21:59:58,912 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 21:59:58,914 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:58,914 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 21:59:58,915 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 21:59:58,916 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 21:59:58,916 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:45945-0x1015dda772d0002, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 21:59:58,916 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 21:59:58,916 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 21:59:58,916 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x1015dda772d0003, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 21:59:58,917 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:58,917 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase20.apache.org,39807,1676325598619, sessionid=0x1015dda772d0000, setting cluster-up flag (Was=false) 2023-02-13 21:59:58,920 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:58,923 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 21:59:58,923 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,39807,1676325598619 2023-02-13 21:59:58,925 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:58,928 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 21:59:58,929 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,39807,1676325598619 2023-02-13 21:59:58,930 WARN [master/jenkins-hbase20:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.hbase-snapshot/.tmp 2023-02-13 21:59:58,932 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-13 21:59:58,932 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 21:59:58,932 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 21:59:58,932 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 21:59:58,932 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 21:59:58,933 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase20:0, corePoolSize=10, maxPoolSize=10 2023-02-13 21:59:58,933 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:58,933 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-13 21:59:58,933 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:58,937 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325628937 2023-02-13 21:59:58,938 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 21:59:58,938 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 21:59:58,938 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 21:59:58,938 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 21:59:58,939 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 21:59:58,939 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 21:59:58,939 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:58,939 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 21:59:58,939 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-13 21:59:58,940 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 21:59:58,940 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 21:59:58,940 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 21:59:58,941 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 21:59:58,943 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 21:59:58,943 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 21:59:58,943 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676325598943,5,FailOnTimeoutGroup] 2023-02-13 21:59:58,944 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676325598943,5,FailOnTimeoutGroup] 2023-02-13 21:59:58,944 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:58,945 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 21:59:58,945 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:58,945 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:58,960 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 21:59:58,961 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 21:59:58,961 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2 2023-02-13 21:59:58,973 INFO [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(952): ClusterId : 1de008e8-3020-43bd-a4f6-073a3ec5ca42 2023-02-13 21:59:58,977 DEBUG [RS:0;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 21:59:58,977 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:58,978 INFO [RS:1;jenkins-hbase20:45945] regionserver.HRegionServer(952): ClusterId : 1de008e8-3020-43bd-a4f6-073a3ec5ca42 2023-02-13 21:59:58,978 INFO [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer(952): ClusterId : 1de008e8-3020-43bd-a4f6-073a3ec5ca42 2023-02-13 21:59:58,978 DEBUG [RS:1;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 21:59:58,979 DEBUG [RS:2;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 21:59:58,979 DEBUG [RS:0;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 21:59:58,979 DEBUG [RS:0;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 21:59:58,979 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 21:59:58,981 DEBUG [RS:1;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 21:59:58,981 DEBUG [RS:1;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 21:59:58,981 DEBUG [RS:2;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 21:59:58,981 DEBUG [RS:2;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 21:59:58,981 DEBUG [RS:0;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 21:59:58,982 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/info 2023-02-13 21:59:58,983 DEBUG [RS:0;jenkins-hbase20:37039] zookeeper.ReadOnlyZKClient(139): Connect 0x0ff110ef to 127.0.0.1:53523 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 21:59:58,983 DEBUG [RS:1;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 21:59:58,983 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 21:59:58,983 DEBUG [RS:2;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 21:59:58,984 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:58,984 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 21:59:58,985 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/rep_barrier 2023-02-13 21:59:58,985 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 21:59:58,986 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:58,986 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 21:59:58,989 DEBUG [RS:2;jenkins-hbase20:40899] zookeeper.ReadOnlyZKClient(139): Connect 0x3a542d03 to 127.0.0.1:53523 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 21:59:58,989 DEBUG [RS:1;jenkins-hbase20:45945] zookeeper.ReadOnlyZKClient(139): Connect 0x384f4d29 to 127.0.0.1:53523 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 21:59:58,991 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/table 2023-02-13 21:59:58,992 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 21:59:58,993 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:58,999 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740 2023-02-13 21:59:59,000 DEBUG [RS:0;jenkins-hbase20:37039] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@136ff61, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 21:59:59,001 DEBUG [RS:1;jenkins-hbase20:45945] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@32b72659, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 21:59:59,001 DEBUG [RS:0;jenkins-hbase20:37039] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@400bf55a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 21:59:59,001 DEBUG [RS:1;jenkins-hbase20:45945] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@77b5ae6b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 21:59:59,001 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740 2023-02-13 21:59:59,006 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 21:59:59,008 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 21:59:59,011 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 21:59:59,012 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60425747, jitterRate=-0.09958620369434357}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 21:59:59,012 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676325598978Initializing all the Stores at 1676325598979 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325598979Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325598979Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325598979Cleaning up temporary data from old regions at 1676325599004 (+25 ms)Cleaning up detritus from prior splits at 1676325599004Region opened successfully at 1676325599012 (+8 ms) 2023-02-13 21:59:59,012 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 21:59:59,012 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 21:59:59,013 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 21:59:59,013 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 21:59:59,013 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 21:59:59,013 DEBUG [RS:0;jenkins-hbase20:37039] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase20:37039 2023-02-13 21:59:59,013 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 21:59:59,013 INFO [RS:0;jenkins-hbase20:37039] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 21:59:59,013 DEBUG [RS:1;jenkins-hbase20:45945] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase20:45945 2023-02-13 21:59:59,013 INFO [RS:0;jenkins-hbase20:37039] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 21:59:59,013 INFO [RS:1;jenkins-hbase20:45945] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 21:59:59,013 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676325599012Disabling compacts and flushes for region at 1676325599012Disabling writes for close at 1676325599013 (+1 ms)Writing region close event to WAL at 1676325599013Closed at 1676325599013 2023-02-13 21:59:59,013 DEBUG [RS:2;jenkins-hbase20:40899] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3493d439, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 21:59:59,013 INFO [RS:1;jenkins-hbase20:45945] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 21:59:59,014 DEBUG [RS:1;jenkins-hbase20:45945] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 21:59:59,014 DEBUG [RS:2;jenkins-hbase20:40899] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@30ac4673, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 21:59:59,015 INFO [RS:1;jenkins-hbase20:45945] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,39807,1676325598619 with isa=jenkins-hbase20.apache.org/148.251.75.209:45945, startcode=1676325598716 2023-02-13 21:59:59,015 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 21:59:59,015 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-13 21:59:59,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 21:59:59,013 DEBUG [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 21:59:59,018 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 21:59:59,015 DEBUG [RS:1;jenkins-hbase20:45945] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 21:59:59,020 INFO [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,39807,1676325598619 with isa=jenkins-hbase20.apache.org/148.251.75.209:37039, startcode=1676325598677 2023-02-13 21:59:59,020 DEBUG [RS:0;jenkins-hbase20:37039] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 21:59:59,021 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 21:59:59,023 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56189, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 21:59:59,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,45945,1676325598716 2023-02-13 21:59:59,025 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42215, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 21:59:59,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 21:59:59,026 DEBUG [RS:1;jenkins-hbase20:45945] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2 2023-02-13 21:59:59,026 DEBUG [RS:1;jenkins-hbase20:45945] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:34137 2023-02-13 21:59:59,026 DEBUG [RS:1;jenkins-hbase20:45945] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 21:59:59,026 DEBUG [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2 2023-02-13 21:59:59,026 DEBUG [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:34137 2023-02-13 21:59:59,026 DEBUG [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 21:59:59,028 DEBUG [RS:2;jenkins-hbase20:40899] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase20:40899 2023-02-13 21:59:59,028 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 21:59:59,028 INFO [RS:2;jenkins-hbase20:40899] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 21:59:59,028 INFO [RS:2;jenkins-hbase20:40899] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 21:59:59,028 DEBUG [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 21:59:59,029 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,37039,1676325598677] 2023-02-13 21:59:59,029 DEBUG [RS:0;jenkins-hbase20:37039] zookeeper.ZKUtil(162): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 21:59:59,029 INFO [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,39807,1676325598619 with isa=jenkins-hbase20.apache.org/148.251.75.209:40899, startcode=1676325598751 2023-02-13 21:59:59,030 WARN [RS:0;jenkins-hbase20:37039] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 21:59:59,029 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,45945,1676325598716] 2023-02-13 21:59:59,030 INFO [RS:0;jenkins-hbase20:37039] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 21:59:59,030 DEBUG [RS:2;jenkins-hbase20:40899] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 21:59:59,030 DEBUG [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 21:59:59,031 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:33007, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.7 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 21:59:59,032 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,40899,1676325598751 2023-02-13 21:59:59,032 DEBUG [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2 2023-02-13 21:59:59,033 DEBUG [RS:1;jenkins-hbase20:45945] zookeeper.ZKUtil(162): regionserver:45945-0x1015dda772d0002, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325598716 2023-02-13 21:59:59,033 DEBUG [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:34137 2023-02-13 21:59:59,033 WARN [RS:1;jenkins-hbase20:45945] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 21:59:59,033 DEBUG [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 21:59:59,033 INFO [RS:1;jenkins-hbase20:45945] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 21:59:59,033 DEBUG [RS:1;jenkins-hbase20:45945] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,45945,1676325598716 2023-02-13 21:59:59,038 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 21:59:59,039 DEBUG [RS:2;jenkins-hbase20:40899] zookeeper.ZKUtil(162): regionserver:40899-0x1015dda772d0003, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,40899,1676325598751 2023-02-13 21:59:59,039 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,40899,1676325598751] 2023-02-13 21:59:59,040 WARN [RS:2;jenkins-hbase20:40899] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 21:59:59,040 INFO [RS:2;jenkins-hbase20:40899] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 21:59:59,040 DEBUG [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,40899,1676325598751 2023-02-13 21:59:59,041 DEBUG [RS:0;jenkins-hbase20:37039] zookeeper.ZKUtil(162): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 21:59:59,042 DEBUG [RS:1;jenkins-hbase20:45945] zookeeper.ZKUtil(162): regionserver:45945-0x1015dda772d0002, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 21:59:59,042 DEBUG [RS:0;jenkins-hbase20:37039] zookeeper.ZKUtil(162): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325598716 2023-02-13 21:59:59,043 DEBUG [RS:1;jenkins-hbase20:45945] zookeeper.ZKUtil(162): regionserver:45945-0x1015dda772d0002, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325598716 2023-02-13 21:59:59,043 DEBUG [RS:0;jenkins-hbase20:37039] zookeeper.ZKUtil(162): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,40899,1676325598751 2023-02-13 21:59:59,043 DEBUG [RS:1;jenkins-hbase20:45945] zookeeper.ZKUtil(162): regionserver:45945-0x1015dda772d0002, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,40899,1676325598751 2023-02-13 21:59:59,045 DEBUG [RS:0;jenkins-hbase20:37039] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 21:59:59,045 DEBUG [RS:1;jenkins-hbase20:45945] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 21:59:59,045 INFO [RS:0;jenkins-hbase20:37039] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 21:59:59,045 DEBUG [RS:2;jenkins-hbase20:40899] zookeeper.ZKUtil(162): regionserver:40899-0x1015dda772d0003, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 21:59:59,045 INFO [RS:1;jenkins-hbase20:45945] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 21:59:59,045 DEBUG [RS:2;jenkins-hbase20:40899] zookeeper.ZKUtil(162): regionserver:40899-0x1015dda772d0003, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325598716 2023-02-13 21:59:59,046 DEBUG [RS:2;jenkins-hbase20:40899] zookeeper.ZKUtil(162): regionserver:40899-0x1015dda772d0003, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,40899,1676325598751 2023-02-13 21:59:59,047 DEBUG [RS:2;jenkins-hbase20:40899] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 21:59:59,047 INFO [RS:0;jenkins-hbase20:37039] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 21:59:59,047 INFO [RS:0;jenkins-hbase20:37039] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 21:59:59,047 INFO [RS:0;jenkins-hbase20:37039] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,048 INFO [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 21:59:59,049 INFO [RS:2;jenkins-hbase20:40899] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 21:59:59,049 INFO [RS:1;jenkins-hbase20:45945] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 21:59:59,051 INFO [RS:0;jenkins-hbase20:37039] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,051 INFO [RS:1;jenkins-hbase20:45945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 21:59:59,051 INFO [RS:1;jenkins-hbase20:45945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,051 DEBUG [RS:0;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,051 DEBUG [RS:0;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,051 INFO [RS:2;jenkins-hbase20:40899] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 21:59:59,051 DEBUG [RS:0;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,051 DEBUG [RS:0;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,051 DEBUG [RS:0;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,051 DEBUG [RS:0;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-13 21:59:59,051 DEBUG [RS:0;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,052 DEBUG [RS:0;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,052 DEBUG [RS:0;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,052 DEBUG [RS:0;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,052 INFO [RS:1;jenkins-hbase20:45945] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 21:59:59,054 INFO [RS:2;jenkins-hbase20:40899] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 21:59:59,054 INFO [RS:2;jenkins-hbase20:40899] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,054 INFO [RS:0;jenkins-hbase20:37039] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,054 INFO [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 21:59:59,054 INFO [RS:0;jenkins-hbase20:37039] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,055 INFO [RS:0;jenkins-hbase20:37039] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,058 INFO [RS:1;jenkins-hbase20:45945] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,058 INFO [RS:2;jenkins-hbase20:40899] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,058 DEBUG [RS:2;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,058 DEBUG [RS:1;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,058 DEBUG [RS:2;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,058 DEBUG [RS:1;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,058 DEBUG [RS:2;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,058 DEBUG [RS:1;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,058 DEBUG [RS:2;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,059 DEBUG [RS:1;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,059 DEBUG [RS:2;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,059 DEBUG [RS:1;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,059 DEBUG [RS:2;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-13 21:59:59,059 DEBUG [RS:1;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-13 21:59:59,059 DEBUG [RS:2;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,059 DEBUG [RS:1;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,059 DEBUG [RS:2;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,059 DEBUG [RS:1;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,059 DEBUG [RS:2;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,059 DEBUG [RS:1;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,059 DEBUG [RS:2;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,059 DEBUG [RS:1;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 21:59:59,067 INFO [RS:2;jenkins-hbase20:40899] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,067 INFO [RS:1;jenkins-hbase20:45945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,067 INFO [RS:2;jenkins-hbase20:40899] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,067 INFO [RS:1;jenkins-hbase20:45945] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,067 INFO [RS:2;jenkins-hbase20:40899] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,067 INFO [RS:1;jenkins-hbase20:45945] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,076 INFO [RS:0;jenkins-hbase20:37039] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 21:59:59,076 INFO [RS:0;jenkins-hbase20:37039] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,37039,1676325598677-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,076 INFO [RS:1;jenkins-hbase20:45945] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 21:59:59,076 INFO [RS:1;jenkins-hbase20:45945] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,45945,1676325598716-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,079 INFO [RS:2;jenkins-hbase20:40899] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 21:59:59,079 INFO [RS:2;jenkins-hbase20:40899] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,40899,1676325598751-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,091 INFO [RS:0;jenkins-hbase20:37039] regionserver.Replication(203): jenkins-hbase20.apache.org,37039,1676325598677 started 2023-02-13 21:59:59,091 INFO [RS:1;jenkins-hbase20:45945] regionserver.Replication(203): jenkins-hbase20.apache.org,45945,1676325598716 started 2023-02-13 21:59:59,091 INFO [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,37039,1676325598677, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:37039, sessionid=0x1015dda772d0001 2023-02-13 21:59:59,091 INFO [RS:1;jenkins-hbase20:45945] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,45945,1676325598716, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:45945, sessionid=0x1015dda772d0002 2023-02-13 21:59:59,091 DEBUG [RS:0;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 21:59:59,091 DEBUG [RS:1;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 21:59:59,091 DEBUG [RS:1;jenkins-hbase20:45945] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,45945,1676325598716 2023-02-13 21:59:59,091 DEBUG [RS:1;jenkins-hbase20:45945] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,45945,1676325598716' 2023-02-13 21:59:59,091 DEBUG [RS:1;jenkins-hbase20:45945] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 21:59:59,091 INFO [RS:2;jenkins-hbase20:40899] regionserver.Replication(203): jenkins-hbase20.apache.org,40899,1676325598751 started 2023-02-13 21:59:59,091 DEBUG [RS:0;jenkins-hbase20:37039] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 21:59:59,092 INFO [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,40899,1676325598751, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:40899, sessionid=0x1015dda772d0003 2023-02-13 21:59:59,092 DEBUG [RS:0;jenkins-hbase20:37039] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,37039,1676325598677' 2023-02-13 21:59:59,092 DEBUG [RS:2;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 21:59:59,092 DEBUG [RS:2;jenkins-hbase20:40899] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,40899,1676325598751 2023-02-13 21:59:59,092 DEBUG [RS:0;jenkins-hbase20:37039] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 21:59:59,092 DEBUG [RS:2;jenkins-hbase20:40899] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,40899,1676325598751' 2023-02-13 21:59:59,092 DEBUG [RS:2;jenkins-hbase20:40899] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 21:59:59,092 DEBUG [RS:1;jenkins-hbase20:45945] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 21:59:59,092 DEBUG [RS:2;jenkins-hbase20:40899] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 21:59:59,092 DEBUG [RS:0;jenkins-hbase20:37039] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 21:59:59,092 DEBUG [RS:1;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 21:59:59,092 DEBUG [RS:1;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 21:59:59,092 DEBUG [RS:1;jenkins-hbase20:45945] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,45945,1676325598716 2023-02-13 21:59:59,092 DEBUG [RS:2;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 21:59:59,092 DEBUG [RS:1;jenkins-hbase20:45945] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,45945,1676325598716' 2023-02-13 21:59:59,093 DEBUG [RS:0;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 21:59:59,093 DEBUG [RS:2;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 21:59:59,093 DEBUG [RS:2;jenkins-hbase20:40899] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,40899,1676325598751 2023-02-13 21:59:59,093 DEBUG [RS:2;jenkins-hbase20:40899] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,40899,1676325598751' 2023-02-13 21:59:59,093 DEBUG [RS:2;jenkins-hbase20:40899] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 21:59:59,093 DEBUG [RS:0;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 21:59:59,093 DEBUG [RS:0;jenkins-hbase20:37039] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 21:59:59,093 DEBUG [RS:1;jenkins-hbase20:45945] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 21:59:59,093 DEBUG [RS:0;jenkins-hbase20:37039] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,37039,1676325598677' 2023-02-13 21:59:59,093 DEBUG [RS:0;jenkins-hbase20:37039] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 21:59:59,093 DEBUG [RS:2;jenkins-hbase20:40899] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 21:59:59,093 DEBUG [RS:0;jenkins-hbase20:37039] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 21:59:59,093 DEBUG [RS:2;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 21:59:59,093 DEBUG [RS:1;jenkins-hbase20:45945] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 21:59:59,093 INFO [RS:2;jenkins-hbase20:40899] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 21:59:59,093 INFO [RS:2;jenkins-hbase20:40899] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 21:59:59,094 DEBUG [RS:0;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 21:59:59,094 INFO [RS:0;jenkins-hbase20:37039] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 21:59:59,094 INFO [RS:0;jenkins-hbase20:37039] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 21:59:59,094 DEBUG [RS:1;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 21:59:59,094 INFO [RS:1;jenkins-hbase20:45945] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 21:59:59,094 INFO [RS:1;jenkins-hbase20:45945] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 21:59:59,173 DEBUG [jenkins-hbase20:39807] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-13 21:59:59,173 DEBUG [jenkins-hbase20:39807] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 21:59:59,176 DEBUG [jenkins-hbase20:39807] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 21:59:59,176 DEBUG [jenkins-hbase20:39807] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 21:59:59,176 DEBUG [jenkins-hbase20:39807] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 21:59:59,176 DEBUG [jenkins-hbase20:39807] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 21:59:59,177 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,37039,1676325598677, state=OPENING 2023-02-13 21:59:59,178 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-13 21:59:59,179 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:59,179 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 21:59:59,179 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,37039,1676325598677}] 2023-02-13 21:59:59,196 INFO [RS:0;jenkins-hbase20:37039] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C37039%2C1676325598677, suffix=, logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,37039,1676325598677, archiveDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs, maxLogs=32 2023-02-13 21:59:59,196 INFO [RS:1;jenkins-hbase20:45945] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C45945%2C1676325598716, suffix=, logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,45945,1676325598716, archiveDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs, maxLogs=32 2023-02-13 21:59:59,196 INFO [RS:2;jenkins-hbase20:40899] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C40899%2C1676325598751, suffix=, logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,40899,1676325598751, archiveDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs, maxLogs=32 2023-02-13 21:59:59,218 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK] 2023-02-13 21:59:59,218 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK] 2023-02-13 21:59:59,218 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK] 2023-02-13 21:59:59,218 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK] 2023-02-13 21:59:59,218 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK] 2023-02-13 21:59:59,220 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK] 2023-02-13 21:59:59,220 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK] 2023-02-13 21:59:59,223 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK] 2023-02-13 21:59:59,223 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK] 2023-02-13 21:59:59,225 INFO [RS:0;jenkins-hbase20:37039] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,37039,1676325598677/jenkins-hbase20.apache.org%2C37039%2C1676325598677.1676325599200 2023-02-13 21:59:59,225 INFO [RS:1;jenkins-hbase20:45945] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,45945,1676325598716/jenkins-hbase20.apache.org%2C45945%2C1676325598716.1676325599201 2023-02-13 21:59:59,225 INFO [RS:2;jenkins-hbase20:40899] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,40899,1676325598751/jenkins-hbase20.apache.org%2C40899%2C1676325598751.1676325599201 2023-02-13 21:59:59,225 DEBUG [RS:0;jenkins-hbase20:37039] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK], DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK], DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK]] 2023-02-13 21:59:59,225 DEBUG [RS:1;jenkins-hbase20:45945] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK], DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK], DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK]] 2023-02-13 21:59:59,226 DEBUG [RS:2;jenkins-hbase20:40899] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK], DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK], DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK]] 2023-02-13 21:59:59,334 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 21:59:59,335 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 21:59:59,336 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:33558, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 21:59:59,342 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 21:59:59,342 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 21:59:59,344 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C37039%2C1676325598677.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,37039,1676325598677, archiveDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs, maxLogs=32 2023-02-13 21:59:59,356 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK] 2023-02-13 21:59:59,357 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK] 2023-02-13 21:59:59,356 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK] 2023-02-13 21:59:59,358 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,37039,1676325598677/jenkins-hbase20.apache.org%2C37039%2C1676325598677.meta.1676325599344.meta 2023-02-13 21:59:59,358 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK], DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK], DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK]] 2023-02-13 21:59:59,359 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 21:59:59,359 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 21:59:59,359 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 21:59:59,359 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 21:59:59,359 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 21:59:59,359 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:59,359 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 21:59:59,359 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 21:59:59,361 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 21:59:59,362 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/info 2023-02-13 21:59:59,362 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/info 2023-02-13 21:59:59,362 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 21:59:59,363 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:59,363 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 21:59:59,364 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/rep_barrier 2023-02-13 21:59:59,364 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/rep_barrier 2023-02-13 21:59:59,365 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 21:59:59,365 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:59,365 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 21:59:59,366 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/table 2023-02-13 21:59:59,366 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/table 2023-02-13 21:59:59,367 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 21:59:59,367 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:59,368 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740 2023-02-13 21:59:59,369 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740 2023-02-13 21:59:59,371 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 21:59:59,373 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 21:59:59,374 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60112343, jitterRate=-0.1042562872171402}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 21:59:59,374 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676325599359Writing region info on filesystem at 1676325599359Initializing all the Stores at 1676325599361 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325599361Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325599361Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325599361Cleaning up temporary data from old regions at 1676325599370 (+9 ms)Cleaning up detritus from prior splits at 1676325599371 (+1 ms)Running coprocessor post-open hooks at 1676325599374 (+3 ms)Region opened successfully at 1676325599374 2023-02-13 21:59:59,375 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676325599334 2023-02-13 21:59:59,381 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 21:59:59,381 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 21:59:59,382 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,37039,1676325598677, state=OPEN 2023-02-13 21:59:59,384 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 21:59:59,384 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 21:59:59,387 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-13 21:59:59,387 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,37039,1676325598677 in 205 msec 2023-02-13 21:59:59,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-13 21:59:59,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 372 msec 2023-02-13 21:59:59,392 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 459 msec 2023-02-13 21:59:59,393 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676325598775, completionTime=-1 2023-02-13 21:59:59,393 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-13 21:59:59,393 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 21:59:59,396 DEBUG [hconnection-0xb319d13-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 21:59:59,397 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:33570, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 21:59:59,399 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 21:59:59,399 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325659399 2023-02-13 21:59:59,399 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325719399 2023-02-13 21:59:59,399 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-13 21:59:59,411 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,39807,1676325598619-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,411 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,39807,1676325598619-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,411 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,39807,1676325598619-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,411 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase20:39807, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,411 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 21:59:59,411 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-13 21:59:59,411 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 21:59:59,413 DEBUG [master/jenkins-hbase20:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-13 21:59:59,413 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-13 21:59:59,415 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 21:59:59,416 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 21:59:59,418 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/hbase/namespace/f02c6dea934906492e781696bf263bb2 2023-02-13 21:59:59,419 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/hbase/namespace/f02c6dea934906492e781696bf263bb2 empty. 2023-02-13 21:59:59,419 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/hbase/namespace/f02c6dea934906492e781696bf263bb2 2023-02-13 21:59:59,419 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-13 21:59:59,442 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-13 21:59:59,443 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => f02c6dea934906492e781696bf263bb2, NAME => 'hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp 2023-02-13 21:59:59,464 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:59,464 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing f02c6dea934906492e781696bf263bb2, disabling compactions & flushes 2023-02-13 21:59:59,464 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 21:59:59,464 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 21:59:59,464 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. after waiting 0 ms 2023-02-13 21:59:59,464 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 21:59:59,464 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 21:59:59,464 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for f02c6dea934906492e781696bf263bb2: Waiting for close lock at 1676325599464Disabling compacts and flushes for region at 1676325599464Disabling writes for close at 1676325599464Writing region close event to WAL at 1676325599464Closed at 1676325599464 2023-02-13 21:59:59,520 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 21:59:59,521 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676325599521"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325599521"}]},"ts":"1676325599521"} 2023-02-13 21:59:59,523 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 21:59:59,525 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 21:59:59,525 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325599525"}]},"ts":"1676325599525"} 2023-02-13 21:59:59,527 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-13 21:59:59,529 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 21:59:59,530 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 21:59:59,530 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 21:59:59,530 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 21:59:59,530 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 21:59:59,531 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f02c6dea934906492e781696bf263bb2, ASSIGN}] 2023-02-13 21:59:59,533 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f02c6dea934906492e781696bf263bb2, ASSIGN 2023-02-13 21:59:59,535 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=f02c6dea934906492e781696bf263bb2, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,40899,1676325598751; forceNewPlan=false, retain=false 2023-02-13 21:59:59,685 INFO [jenkins-hbase20:39807] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 21:59:59,686 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f02c6dea934906492e781696bf263bb2, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,40899,1676325598751 2023-02-13 21:59:59,686 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676325599686"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325599686"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325599686"}]},"ts":"1676325599686"} 2023-02-13 21:59:59,689 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure f02c6dea934906492e781696bf263bb2, server=jenkins-hbase20.apache.org,40899,1676325598751}] 2023-02-13 21:59:59,842 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,40899,1676325598751 2023-02-13 21:59:59,842 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 21:59:59,844 INFO [RS-EventLoopGroup-12-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:48368, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 21:59:59,848 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 21:59:59,848 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f02c6dea934906492e781696bf263bb2, NAME => 'hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2.', STARTKEY => '', ENDKEY => ''} 2023-02-13 21:59:59,848 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace f02c6dea934906492e781696bf263bb2 2023-02-13 21:59:59,848 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 21:59:59,848 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for f02c6dea934906492e781696bf263bb2 2023-02-13 21:59:59,848 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for f02c6dea934906492e781696bf263bb2 2023-02-13 21:59:59,850 INFO [StoreOpener-f02c6dea934906492e781696bf263bb2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region f02c6dea934906492e781696bf263bb2 2023-02-13 21:59:59,852 DEBUG [StoreOpener-f02c6dea934906492e781696bf263bb2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2/info 2023-02-13 21:59:59,852 DEBUG [StoreOpener-f02c6dea934906492e781696bf263bb2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2/info 2023-02-13 21:59:59,852 INFO [StoreOpener-f02c6dea934906492e781696bf263bb2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f02c6dea934906492e781696bf263bb2 columnFamilyName info 2023-02-13 21:59:59,853 INFO [StoreOpener-f02c6dea934906492e781696bf263bb2-1] regionserver.HStore(310): Store=f02c6dea934906492e781696bf263bb2/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 21:59:59,854 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2 2023-02-13 21:59:59,854 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2 2023-02-13 21:59:59,857 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for f02c6dea934906492e781696bf263bb2 2023-02-13 21:59:59,860 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 21:59:59,861 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened f02c6dea934906492e781696bf263bb2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69326238, jitterRate=0.03304144740104675}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 21:59:59,861 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for f02c6dea934906492e781696bf263bb2: Running coprocessor pre-open hook at 1676325599849Writing region info on filesystem at 1676325599849Initializing all the Stores at 1676325599850 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325599850Cleaning up temporary data from old regions at 1676325599854 (+4 ms)Cleaning up detritus from prior splits at 1676325599855 (+1 ms)Running coprocessor post-open hooks at 1676325599861 (+6 ms)Region opened successfully at 1676325599861 2023-02-13 21:59:59,862 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2., pid=6, masterSystemTime=1676325599842 2023-02-13 21:59:59,865 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 21:59:59,866 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 21:59:59,867 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f02c6dea934906492e781696bf263bb2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,40899,1676325598751 2023-02-13 21:59:59,867 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676325599867"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325599867"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325599867"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325599867"}]},"ts":"1676325599867"} 2023-02-13 21:59:59,871 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-13 21:59:59,872 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure f02c6dea934906492e781696bf263bb2, server=jenkins-hbase20.apache.org,40899,1676325598751 in 180 msec 2023-02-13 21:59:59,874 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-13 21:59:59,874 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=f02c6dea934906492e781696bf263bb2, ASSIGN in 340 msec 2023-02-13 21:59:59,875 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 21:59:59,875 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325599875"}]},"ts":"1676325599875"} 2023-02-13 21:59:59,876 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-13 21:59:59,879 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 21:59:59,881 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 468 msec 2023-02-13 21:59:59,919 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-13 21:59:59,920 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-13 21:59:59,920 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 21:59:59,923 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 21:59:59,924 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:48372, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 21:59:59,926 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-13 21:59:59,936 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 21:59:59,940 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 13 msec 2023-02-13 21:59:59,948 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-13 21:59:59,958 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 21:59:59,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 12 msec 2023-02-13 21:59:59,972 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 21:59:59,973 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 21:59:59,974 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 1.195sec 2023-02-13 21:59:59,974 INFO [master/jenkins-hbase20:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 21:59:59,974 INFO [master/jenkins-hbase20:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 21:59:59,974 INFO [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 21:59:59,974 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,39807,1676325598619-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 21:59:59,974 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,39807,1676325598619-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 21:59:59,976 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 22:00:00,019 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ReadOnlyZKClient(139): Connect 0x3bdb158c to 127.0.0.1:53523 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 22:00:00,025 DEBUG [Listener at localhost.localdomain/39067] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5fc63ab0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 22:00:00,027 DEBUG [hconnection-0x29489a59-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 22:00:00,028 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:33574, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 22:00:00,030 INFO [Listener at localhost.localdomain/39067] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase20.apache.org,39807,1676325598619 2023-02-13 22:00:00,032 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-13 22:00:00,032 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:00:00,033 INFO [Listener at localhost.localdomain/39067] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-02-13 22:00:00,033 INFO [Listener at localhost.localdomain/39067] master.TestRetainAssignmentOnRestart(212): Creating tables 2023-02-13 22:00:00,036 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 22:00:00,038 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52612, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 22:00:00,039 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.HMaster$4(2102): Client=jenkins//148.251.75.209 create 'restartTableOne', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 22:00:00,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=restartTableOne 2023-02-13 22:00:00,044 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=restartTableOne execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 22:00:00,044 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.MasterRpcServices(697): Client=jenkins//148.251.75.209 procedure request for creating table: namespace: "default" qualifier: "restartTableOne" procId is: 9 2023-02-13 22:00:00,045 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=restartTableOne execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 22:00:00,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 22:00:00,047 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/default/restartTableOne/78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:00,047 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/default/restartTableOne/78b5804c7ea67fd4601a55f3e6591a8d empty. 2023-02-13 22:00:00,048 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/default/restartTableOne/78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:00,048 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived restartTableOne regions 2023-02-13 22:00:00,063 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/default/restartTableOne/.tabledesc/.tableinfo.0000000001 2023-02-13 22:00:00,064 INFO [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(7671): creating {ENCODED => 78b5804c7ea67fd4601a55f3e6591a8d, NAME => 'restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='restartTableOne', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp 2023-02-13 22:00:00,079 DEBUG [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(865): Instantiated restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:00,079 DEBUG [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(1603): Closing 78b5804c7ea67fd4601a55f3e6591a8d, disabling compactions & flushes 2023-02-13 22:00:00,079 INFO [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(1625): Closing region restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:00,079 DEBUG [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:00,079 DEBUG [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(1713): Acquired close lock on restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. after waiting 0 ms 2023-02-13 22:00:00,079 DEBUG [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(1723): Updates disabled for region restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:00,079 INFO [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(1837): Closed restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:00,079 DEBUG [RegionOpenAndInit-restartTableOne-pool-0] regionserver.HRegion(1557): Region close journal for 78b5804c7ea67fd4601a55f3e6591a8d: Waiting for close lock at 1676325600079Disabling compacts and flushes for region at 1676325600079Disabling writes for close at 1676325600079Writing region close event to WAL at 1676325600079Closed at 1676325600079 2023-02-13 22:00:00,082 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=restartTableOne execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 22:00:00,083 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325600083"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325600083"}]},"ts":"1676325600083"} 2023-02-13 22:00:00,085 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 22:00:00,086 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=restartTableOne execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 22:00:00,086 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"restartTableOne","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325600086"}]},"ts":"1676325600086"} 2023-02-13 22:00:00,088 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=restartTableOne, state=ENABLING in hbase:meta 2023-02-13 22:00:00,090 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:00:00,091 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:00:00,091 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:00:00,091 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:00:00,091 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 22:00:00,091 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableOne, region=78b5804c7ea67fd4601a55f3e6591a8d, ASSIGN}] 2023-02-13 22:00:00,093 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableOne, region=78b5804c7ea67fd4601a55f3e6591a8d, ASSIGN 2023-02-13 22:00:00,094 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=restartTableOne, region=78b5804c7ea67fd4601a55f3e6591a8d, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,37039,1676325598677; forceNewPlan=false, retain=false 2023-02-13 22:00:00,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 22:00:00,244 INFO [jenkins-hbase20:39807] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:00:00,245 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=78b5804c7ea67fd4601a55f3e6591a8d, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 22:00:00,246 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325600245"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325600245"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325600245"}]},"ts":"1676325600245"} 2023-02-13 22:00:00,248 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 78b5804c7ea67fd4601a55f3e6591a8d, server=jenkins-hbase20.apache.org,37039,1676325598677}] 2023-02-13 22:00:00,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 22:00:00,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:00,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 78b5804c7ea67fd4601a55f3e6591a8d, NAME => 'restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:00,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table restartTableOne 78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:00,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:00,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:00,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:00,408 INFO [StoreOpener-78b5804c7ea67fd4601a55f3e6591a8d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family of region 78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:00,409 DEBUG [StoreOpener-78b5804c7ea67fd4601a55f3e6591a8d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableOne/78b5804c7ea67fd4601a55f3e6591a8d/family 2023-02-13 22:00:00,409 DEBUG [StoreOpener-78b5804c7ea67fd4601a55f3e6591a8d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableOne/78b5804c7ea67fd4601a55f3e6591a8d/family 2023-02-13 22:00:00,410 INFO [StoreOpener-78b5804c7ea67fd4601a55f3e6591a8d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 78b5804c7ea67fd4601a55f3e6591a8d columnFamilyName family 2023-02-13 22:00:00,410 INFO [StoreOpener-78b5804c7ea67fd4601a55f3e6591a8d-1] regionserver.HStore(310): Store=78b5804c7ea67fd4601a55f3e6591a8d/family, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:00,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableOne/78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:00,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableOne/78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:00,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:00,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableOne/78b5804c7ea67fd4601a55f3e6591a8d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:00:00,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 78b5804c7ea67fd4601a55f3e6591a8d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67243453, jitterRate=0.002005532383918762}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 22:00:00,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 78b5804c7ea67fd4601a55f3e6591a8d: Running coprocessor pre-open hook at 1676325600406Writing region info on filesystem at 1676325600406Initializing all the Stores at 1676325600407 (+1 ms)Instantiating store for column family {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325600407Cleaning up temporary data from old regions at 1676325600412 (+5 ms)Cleaning up detritus from prior splits at 1676325600412Running coprocessor post-open hooks at 1676325600417 (+5 ms)Region opened successfully at 1676325600417 2023-02-13 22:00:00,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d., pid=11, masterSystemTime=1676325600401 2023-02-13 22:00:00,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:00,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:00,421 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=78b5804c7ea67fd4601a55f3e6591a8d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 22:00:00,421 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325600421"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325600421"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325600421"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325600421"}]},"ts":"1676325600421"} 2023-02-13 22:00:00,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-13 22:00:00,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 78b5804c7ea67fd4601a55f3e6591a8d, server=jenkins-hbase20.apache.org,37039,1676325598677 in 175 msec 2023-02-13 22:00:00,427 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-13 22:00:00,427 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=restartTableOne, region=78b5804c7ea67fd4601a55f3e6591a8d, ASSIGN in 334 msec 2023-02-13 22:00:00,428 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=restartTableOne execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 22:00:00,428 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"restartTableOne","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325600428"}]},"ts":"1676325600428"} 2023-02-13 22:00:00,429 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=restartTableOne, state=ENABLED in hbase:meta 2023-02-13 22:00:00,432 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=restartTableOne execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 22:00:00,433 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=restartTableOne in 392 msec 2023-02-13 22:00:00,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 22:00:00,651 INFO [Listener at localhost.localdomain/39067] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:restartTableOne, procId: 9 completed 2023-02-13 22:00:00,651 DEBUG [Listener at localhost.localdomain/39067] hbase.HBaseTestingUtility(3430): Waiting until all regions of table restartTableOne get assigned. Timeout = 60000ms 2023-02-13 22:00:00,651 INFO [Listener at localhost.localdomain/39067] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:00,654 INFO [Listener at localhost.localdomain/39067] hbase.HBaseTestingUtility(3484): All regions for table restartTableOne assigned to meta. Checking AM states. 2023-02-13 22:00:00,654 INFO [Listener at localhost.localdomain/39067] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:00,654 INFO [Listener at localhost.localdomain/39067] hbase.HBaseTestingUtility(3504): All regions for table restartTableOne assigned. 2023-02-13 22:00:00,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.HMaster$4(2102): Client=jenkins//148.251.75.209 create 'restartTableTwo', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 22:00:00,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=restartTableTwo 2023-02-13 22:00:00,659 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=restartTableTwo execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 22:00:00,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.MasterRpcServices(697): Client=jenkins//148.251.75.209 procedure request for creating table: namespace: "default" qualifier: "restartTableTwo" procId is: 12 2023-02-13 22:00:00,660 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=restartTableTwo execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 22:00:00,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.MasterRpcServices(1227): Checking to see if procedure is done pid=12 2023-02-13 22:00:00,662 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/default/restartTableTwo/a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:00,663 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/default/restartTableTwo/a9372b7742e56bdddc1c8be728e71577 empty. 2023-02-13 22:00:00,663 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/default/restartTableTwo/a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:00,663 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived restartTableTwo regions 2023-02-13 22:00:00,677 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/default/restartTableTwo/.tabledesc/.tableinfo.0000000001 2023-02-13 22:00:00,678 INFO [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(7671): creating {ENCODED => a9372b7742e56bdddc1c8be728e71577, NAME => 'restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='restartTableTwo', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp 2023-02-13 22:00:00,689 DEBUG [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(865): Instantiated restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:00,690 DEBUG [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(1603): Closing a9372b7742e56bdddc1c8be728e71577, disabling compactions & flushes 2023-02-13 22:00:00,690 INFO [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(1625): Closing region restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:00,690 DEBUG [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:00,690 DEBUG [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(1713): Acquired close lock on restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. after waiting 0 ms 2023-02-13 22:00:00,690 DEBUG [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(1723): Updates disabled for region restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:00,690 INFO [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(1837): Closed restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:00,690 DEBUG [RegionOpenAndInit-restartTableTwo-pool-0] regionserver.HRegion(1557): Region close journal for a9372b7742e56bdddc1c8be728e71577: Waiting for close lock at 1676325600690Disabling compacts and flushes for region at 1676325600690Disabling writes for close at 1676325600690Writing region close event to WAL at 1676325600690Closed at 1676325600690 2023-02-13 22:00:00,693 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=restartTableTwo execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 22:00:00,694 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325600694"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325600694"}]},"ts":"1676325600694"} 2023-02-13 22:00:00,696 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 22:00:00,697 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=restartTableTwo execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 22:00:00,697 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"restartTableTwo","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325600697"}]},"ts":"1676325600697"} 2023-02-13 22:00:00,699 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=restartTableTwo, state=ENABLING in hbase:meta 2023-02-13 22:00:00,701 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:00:00,701 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:00:00,702 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:00:00,702 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:00:00,702 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 22:00:00,702 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableTwo, region=a9372b7742e56bdddc1c8be728e71577, ASSIGN}] 2023-02-13 22:00:00,704 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableTwo, region=a9372b7742e56bdddc1c8be728e71577, ASSIGN 2023-02-13 22:00:00,705 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=restartTableTwo, region=a9372b7742e56bdddc1c8be728e71577, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,37039,1676325598677; forceNewPlan=false, retain=false 2023-02-13 22:00:00,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.MasterRpcServices(1227): Checking to see if procedure is done pid=12 2023-02-13 22:00:00,789 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-02-13 22:00:00,855 INFO [jenkins-hbase20:39807] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:00:00,856 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=a9372b7742e56bdddc1c8be728e71577, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 22:00:00,856 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325600856"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325600856"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325600856"}]},"ts":"1676325600856"} 2023-02-13 22:00:00,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure a9372b7742e56bdddc1c8be728e71577, server=jenkins-hbase20.apache.org,37039,1676325598677}] 2023-02-13 22:00:00,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.MasterRpcServices(1227): Checking to see if procedure is done pid=12 2023-02-13 22:00:01,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:01,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a9372b7742e56bdddc1c8be728e71577, NAME => 'restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:01,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table restartTableTwo a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:01,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:01,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:01,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:01,017 INFO [StoreOpener-a9372b7742e56bdddc1c8be728e71577-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family of region a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:01,019 DEBUG [StoreOpener-a9372b7742e56bdddc1c8be728e71577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableTwo/a9372b7742e56bdddc1c8be728e71577/family 2023-02-13 22:00:01,019 DEBUG [StoreOpener-a9372b7742e56bdddc1c8be728e71577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableTwo/a9372b7742e56bdddc1c8be728e71577/family 2023-02-13 22:00:01,019 INFO [StoreOpener-a9372b7742e56bdddc1c8be728e71577-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9372b7742e56bdddc1c8be728e71577 columnFamilyName family 2023-02-13 22:00:01,020 INFO [StoreOpener-a9372b7742e56bdddc1c8be728e71577-1] regionserver.HStore(310): Store=a9372b7742e56bdddc1c8be728e71577/family, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:01,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableTwo/a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:01,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableTwo/a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:01,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:01,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableTwo/a9372b7742e56bdddc1c8be728e71577/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:00:01,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened a9372b7742e56bdddc1c8be728e71577; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72576420, jitterRate=0.08147293329238892}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 22:00:01,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for a9372b7742e56bdddc1c8be728e71577: Running coprocessor pre-open hook at 1676325601016Writing region info on filesystem at 1676325601016Initializing all the Stores at 1676325601017 (+1 ms)Instantiating store for column family {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325601017Cleaning up temporary data from old regions at 1676325601021 (+4 ms)Cleaning up detritus from prior splits at 1676325601022 (+1 ms)Running coprocessor post-open hooks at 1676325601027 (+5 ms)Region opened successfully at 1676325601027 2023-02-13 22:00:01,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577., pid=14, masterSystemTime=1676325601011 2023-02-13 22:00:01,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:01,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:01,031 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=a9372b7742e56bdddc1c8be728e71577, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 22:00:01,031 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325601031"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325601031"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325601031"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325601031"}]},"ts":"1676325601031"} 2023-02-13 22:00:01,035 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-13 22:00:01,035 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure a9372b7742e56bdddc1c8be728e71577, server=jenkins-hbase20.apache.org,37039,1676325598677 in 175 msec 2023-02-13 22:00:01,038 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-13 22:00:01,038 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=restartTableTwo, region=a9372b7742e56bdddc1c8be728e71577, ASSIGN in 333 msec 2023-02-13 22:00:01,039 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=restartTableTwo execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 22:00:01,039 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"restartTableTwo","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325601039"}]},"ts":"1676325601039"} 2023-02-13 22:00:01,041 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=restartTableTwo, state=ENABLED in hbase:meta 2023-02-13 22:00:01,043 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=restartTableTwo execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 22:00:01,045 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; CreateTableProcedure table=restartTableTwo in 387 msec 2023-02-13 22:00:01,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.MasterRpcServices(1227): Checking to see if procedure is done pid=12 2023-02-13 22:00:01,266 INFO [Listener at localhost.localdomain/39067] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:restartTableTwo, procId: 12 completed 2023-02-13 22:00:01,266 DEBUG [Listener at localhost.localdomain/39067] hbase.HBaseTestingUtility(3430): Waiting until all regions of table restartTableTwo get assigned. Timeout = 60000ms 2023-02-13 22:00:01,266 INFO [Listener at localhost.localdomain/39067] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:01,269 INFO [Listener at localhost.localdomain/39067] hbase.HBaseTestingUtility(3484): All regions for table restartTableTwo assigned to meta. Checking AM states. 2023-02-13 22:00:01,269 INFO [Listener at localhost.localdomain/39067] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:01,269 INFO [Listener at localhost.localdomain/39067] hbase.HBaseTestingUtility(3504): All regions for table restartTableTwo assigned. 2023-02-13 22:00:01,271 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.HMaster$4(2102): Client=jenkins//148.251.75.209 create 'restartTableThree', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 22:00:01,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=restartTableThree 2023-02-13 22:00:01,275 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=restartTableThree execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 22:00:01,275 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.MasterRpcServices(697): Client=jenkins//148.251.75.209 procedure request for creating table: namespace: "default" qualifier: "restartTableThree" procId is: 15 2023-02-13 22:00:01,276 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=restartTableThree execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 22:00:01,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.MasterRpcServices(1227): Checking to see if procedure is done pid=15 2023-02-13 22:00:01,278 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/default/restartTableThree/f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:01,278 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/default/restartTableThree/f3962bf3439d5c7af4e285d7c3a11f11 empty. 2023-02-13 22:00:01,279 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/default/restartTableThree/f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:01,279 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived restartTableThree regions 2023-02-13 22:00:01,285 WARN [IPC Server handler 2 on default port 34137] blockmanagement.BlockPlacementPolicyDefault(446): Failed to place enough replicas, still in need of 1 to reach 3 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy and org.apache.hadoop.net.NetworkTopology 2023-02-13 22:00:01,286 WARN [IPC Server handler 2 on default port 34137] protocol.BlockStoragePolicy(161): Failed to place enough replicas: expected size is 1 but only 0 storage types can be selected (replication=3, selected=[], unavailable=[DISK], removed=[DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}) 2023-02-13 22:00:01,286 WARN [IPC Server handler 2 on default port 34137] blockmanagement.BlockPlacementPolicyDefault(446): Failed to place enough replicas, still in need of 1 to reach 3 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) All required storage types are unavailable: unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]} 2023-02-13 22:00:01,293 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp/data/default/restartTableThree/.tabledesc/.tableinfo.0000000001 2023-02-13 22:00:01,294 INFO [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(7671): creating {ENCODED => f3962bf3439d5c7af4e285d7c3a11f11, NAME => 'restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='restartTableThree', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.tmp 2023-02-13 22:00:01,298 WARN [IPC Server handler 2 on default port 34137] blockmanagement.BlockPlacementPolicyDefault(446): Failed to place enough replicas, still in need of 1 to reach 3 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy and org.apache.hadoop.net.NetworkTopology 2023-02-13 22:00:01,298 WARN [IPC Server handler 2 on default port 34137] protocol.BlockStoragePolicy(161): Failed to place enough replicas: expected size is 1 but only 0 storage types can be selected (replication=3, selected=[], unavailable=[DISK], removed=[DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}) 2023-02-13 22:00:01,298 WARN [IPC Server handler 2 on default port 34137] blockmanagement.BlockPlacementPolicyDefault(446): Failed to place enough replicas, still in need of 1 to reach 3 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) All required storage types are unavailable: unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]} 2023-02-13 22:00:01,306 DEBUG [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(865): Instantiated restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:01,306 DEBUG [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(1603): Closing f3962bf3439d5c7af4e285d7c3a11f11, disabling compactions & flushes 2023-02-13 22:00:01,306 INFO [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(1625): Closing region restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:01,306 DEBUG [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:01,306 DEBUG [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(1713): Acquired close lock on restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. after waiting 0 ms 2023-02-13 22:00:01,306 DEBUG [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(1723): Updates disabled for region restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:01,306 INFO [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(1837): Closed restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:01,306 DEBUG [RegionOpenAndInit-restartTableThree-pool-0] regionserver.HRegion(1557): Region close journal for f3962bf3439d5c7af4e285d7c3a11f11: Waiting for close lock at 1676325601306Disabling compacts and flushes for region at 1676325601306Disabling writes for close at 1676325601306Writing region close event to WAL at 1676325601306Closed at 1676325601306 2023-02-13 22:00:01,309 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=restartTableThree execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 22:00:01,310 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11.","families":{"info":[{"qualifier":"regioninfo","vlen":51,"tag":[],"timestamp":"1676325601310"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676325601310"}]},"ts":"1676325601310"} 2023-02-13 22:00:01,312 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 22:00:01,313 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=restartTableThree execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 22:00:01,313 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"restartTableThree","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325601313"}]},"ts":"1676325601313"} 2023-02-13 22:00:01,315 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=restartTableThree, state=ENABLING in hbase:meta 2023-02-13 22:00:01,317 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-13 22:00:01,318 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 22:00:01,318 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 22:00:01,318 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 22:00:01,318 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 22:00:01,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableThree, region=f3962bf3439d5c7af4e285d7c3a11f11, ASSIGN}] 2023-02-13 22:00:01,321 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableThree, region=f3962bf3439d5c7af4e285d7c3a11f11, ASSIGN 2023-02-13 22:00:01,322 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=restartTableThree, region=f3962bf3439d5c7af4e285d7c3a11f11, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,37039,1676325598677; forceNewPlan=false, retain=false 2023-02-13 22:00:01,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.MasterRpcServices(1227): Checking to see if procedure is done pid=15 2023-02-13 22:00:01,473 INFO [jenkins-hbase20:39807] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:00:01,474 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=f3962bf3439d5c7af4e285d7c3a11f11, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 22:00:01,474 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11.","families":{"info":[{"qualifier":"regioninfo","vlen":51,"tag":[],"timestamp":"1676325601474"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325601474"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325601474"}]},"ts":"1676325601474"} 2023-02-13 22:00:01,477 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure f3962bf3439d5c7af4e285d7c3a11f11, server=jenkins-hbase20.apache.org,37039,1676325598677}] 2023-02-13 22:00:01,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.MasterRpcServices(1227): Checking to see if procedure is done pid=15 2023-02-13 22:00:01,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:01,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f3962bf3439d5c7af4e285d7c3a11f11, NAME => 'restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:01,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table restartTableThree f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:01,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:01,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:01,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:01,635 INFO [StoreOpener-f3962bf3439d5c7af4e285d7c3a11f11-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family of region f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:01,637 DEBUG [StoreOpener-f3962bf3439d5c7af4e285d7c3a11f11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableThree/f3962bf3439d5c7af4e285d7c3a11f11/family 2023-02-13 22:00:01,637 DEBUG [StoreOpener-f3962bf3439d5c7af4e285d7c3a11f11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableThree/f3962bf3439d5c7af4e285d7c3a11f11/family 2023-02-13 22:00:01,637 INFO [StoreOpener-f3962bf3439d5c7af4e285d7c3a11f11-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f3962bf3439d5c7af4e285d7c3a11f11 columnFamilyName family 2023-02-13 22:00:01,638 INFO [StoreOpener-f3962bf3439d5c7af4e285d7c3a11f11-1] regionserver.HStore(310): Store=f3962bf3439d5c7af4e285d7c3a11f11/family, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:01,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableThree/f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:01,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableThree/f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:01,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:01,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableThree/f3962bf3439d5c7af4e285d7c3a11f11/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 22:00:01,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened f3962bf3439d5c7af4e285d7c3a11f11; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61568081, jitterRate=-0.0825641006231308}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 22:00:01,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for f3962bf3439d5c7af4e285d7c3a11f11: Running coprocessor pre-open hook at 1676325601634Writing region info on filesystem at 1676325601634Initializing all the Stores at 1676325601635 (+1 ms)Instantiating store for column family {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325601635Cleaning up temporary data from old regions at 1676325601639 (+4 ms)Cleaning up detritus from prior splits at 1676325601640 (+1 ms)Running coprocessor post-open hooks at 1676325601645 (+5 ms)Region opened successfully at 1676325601645 2023-02-13 22:00:01,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11., pid=17, masterSystemTime=1676325601629 2023-02-13 22:00:01,648 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:01,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:01,648 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=f3962bf3439d5c7af4e285d7c3a11f11, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 22:00:01,649 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11.","families":{"info":[{"qualifier":"regioninfo","vlen":51,"tag":[],"timestamp":"1676325601648"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325601648"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325601648"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325601648"}]},"ts":"1676325601648"} 2023-02-13 22:00:01,654 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-02-13 22:00:01,654 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure f3962bf3439d5c7af4e285d7c3a11f11, server=jenkins-hbase20.apache.org,37039,1676325598677 in 175 msec 2023-02-13 22:00:01,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-02-13 22:00:01,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=restartTableThree, region=f3962bf3439d5c7af4e285d7c3a11f11, ASSIGN in 335 msec 2023-02-13 22:00:01,659 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=restartTableThree execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 22:00:01,660 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"restartTableThree","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676325601660"}]},"ts":"1676325601660"} 2023-02-13 22:00:01,662 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=restartTableThree, state=ENABLED in hbase:meta 2023-02-13 22:00:01,665 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=restartTableThree execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 22:00:01,671 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; CreateTableProcedure table=restartTableThree in 394 msec 2023-02-13 22:00:01,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39807] master.MasterRpcServices(1227): Checking to see if procedure is done pid=15 2023-02-13 22:00:01,882 INFO [Listener at localhost.localdomain/39067] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:restartTableThree, procId: 15 completed 2023-02-13 22:00:01,882 DEBUG [Listener at localhost.localdomain/39067] hbase.HBaseTestingUtility(3430): Waiting until all regions of table restartTableThree get assigned. Timeout = 60000ms 2023-02-13 22:00:01,882 INFO [Listener at localhost.localdomain/39067] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:01,886 INFO [Listener at localhost.localdomain/39067] hbase.HBaseTestingUtility(3484): All regions for table restartTableThree assigned to meta. Checking AM states. 2023-02-13 22:00:01,886 INFO [Listener at localhost.localdomain/39067] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:01,886 INFO [Listener at localhost.localdomain/39067] hbase.HBaseTestingUtility(3504): All regions for table restartTableThree assigned. 2023-02-13 22:00:01,886 INFO [Listener at localhost.localdomain/39067] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:01,890 INFO [Listener at localhost.localdomain/39067] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:01,895 INFO [Listener at localhost.localdomain/39067] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:01,898 INFO [Listener at localhost.localdomain/39067] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:01,898 INFO [Listener at localhost.localdomain/39067] master.SnapshotOfRegionAssignmentFromMeta(102): Start to scan the hbase:meta for the current region assignment snappshot 2023-02-13 22:00:01,901 INFO [Listener at localhost.localdomain/39067] master.SnapshotOfRegionAssignmentFromMeta(164): Finished to scan the hbase:meta for the current region assignmentsnapshot 2023-02-13 22:00:01,902 INFO [Listener at localhost.localdomain/39067] master.TestRetainAssignmentOnRestart(88): Shutting down HBase cluster 2023-02-13 22:00:01,902 INFO [Listener at localhost.localdomain/39067] hbase.MiniHBaseCluster(631): Stopping Thread[M:0;jenkins-hbase20:39807,5,FailOnTimeoutGroup] 2023-02-13 22:00:01,902 INFO [Listener at localhost.localdomain/39067] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,39807,1676325598619' ***** 2023-02-13 22:00:01,902 INFO [Listener at localhost.localdomain/39067] regionserver.HRegionServer(2310): STOPPED: Stopping master 0 2023-02-13 22:00:01,903 DEBUG [M:0;jenkins-hbase20:39807] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6f1c267, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 22:00:01,903 INFO [M:0;jenkins-hbase20:39807] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,39807,1676325598619 2023-02-13 22:00:01,903 DEBUG [M:0;jenkins-hbase20:39807] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4330f13a to 127.0.0.1:53523 2023-02-13 22:00:01,904 DEBUG [M:0;jenkins-hbase20:39807] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:01,904 INFO [M:0;jenkins-hbase20:39807] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,39807,1676325598619; all regions closed. 2023-02-13 22:00:01,904 DEBUG [M:0;jenkins-hbase20:39807] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:01,904 DEBUG [M:0;jenkins-hbase20:39807] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 22:00:01,904 DEBUG [M:0;jenkins-hbase20:39807] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 22:00:01,904 INFO [M:0;jenkins-hbase20:39807] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 22:00:01,904 INFO [M:0;jenkins-hbase20:39807] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 22:00:01,904 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 22:00:01,904 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676325598943] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676325598943,5,FailOnTimeoutGroup] 2023-02-13 22:00:01,904 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676325598943] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676325598943,5,FailOnTimeoutGroup] 2023-02-13 22:00:01,905 INFO [M:0;jenkins-hbase20:39807] hbase.ChoreService(369): Chore service for: master/jenkins-hbase20:0 had [] on shutdown 2023-02-13 22:00:01,905 DEBUG [M:0;jenkins-hbase20:39807] master.HMaster(1502): Stopping service threads 2023-02-13 22:00:01,905 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 22:00:01,906 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:45945-0x1015dda772d0002, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 22:00:01,906 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 22:00:01,906 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:00:01,905 INFO [M:0;jenkins-hbase20:39807] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 22:00:01,907 DEBUG [Listener at localhost.localdomain/39067] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-13 22:00:01,905 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x1015dda772d0003, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 22:00:01,907 INFO [Listener at localhost.localdomain/39067] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,37039,1676325598677' ***** 2023-02-13 22:00:01,907 INFO [M:0;jenkins-hbase20:39807] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:01,908 INFO [Listener at localhost.localdomain/39067] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 22:00:01,908 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 22:00:01,908 INFO [Listener at localhost.localdomain/39067] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,45945,1676325598716' ***** 2023-02-13 22:00:01,908 INFO [Listener at localhost.localdomain/39067] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 22:00:01,908 INFO [RS:0;jenkins-hbase20:37039] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 22:00:01,909 INFO [Listener at localhost.localdomain/39067] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,40899,1676325598751' ***** 2023-02-13 22:00:01,908 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45945-0x1015dda772d0002, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 22:00:01,909 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 22:00:01,909 INFO [RS:0;jenkins-hbase20:37039] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 22:00:01,909 INFO [RS:0;jenkins-hbase20:37039] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 22:00:01,909 INFO [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(3304): Received CLOSE for 78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:01,909 INFO [RS:1;jenkins-hbase20:45945] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 22:00:01,909 INFO [Listener at localhost.localdomain/39067] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 22:00:01,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 78b5804c7ea67fd4601a55f3e6591a8d, disabling compactions & flushes 2023-02-13 22:00:01,910 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:01,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:01,911 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. after waiting 0 ms 2023-02-13 22:00:01,911 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:01,909 ERROR [M:0;jenkins-hbase20:39807] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-13 22:00:01,911 INFO [M:0;jenkins-hbase20:39807] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:01,911 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 22:00:01,910 INFO [RS:2;jenkins-hbase20:40899] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 22:00:01,910 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 22:00:01,910 INFO [RS:1;jenkins-hbase20:45945] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 22:00:01,910 INFO [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(3304): Received CLOSE for f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:01,912 INFO [RS:1;jenkins-hbase20:45945] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 22:00:01,912 INFO [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(3304): Received CLOSE for a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:01,912 INFO [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 22:00:01,912 DEBUG [RS:0;jenkins-hbase20:37039] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0ff110ef to 127.0.0.1:53523 2023-02-13 22:00:01,912 DEBUG [RS:0;jenkins-hbase20:37039] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:01,911 INFO [RS:2;jenkins-hbase20:40899] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 22:00:01,911 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 22:00:01,912 INFO [RS:2;jenkins-hbase20:40899] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 22:00:01,912 INFO [RS:0;jenkins-hbase20:37039] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 22:00:01,913 INFO [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer(3304): Received CLOSE for f02c6dea934906492e781696bf263bb2 2023-02-13 22:00:01,912 INFO [RS:1;jenkins-hbase20:45945] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,45945,1676325598716 2023-02-13 22:00:01,913 INFO [RS:0;jenkins-hbase20:37039] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 22:00:01,913 DEBUG [RS:1;jenkins-hbase20:45945] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x384f4d29 to 127.0.0.1:53523 2023-02-13 22:00:01,913 INFO [RS:0;jenkins-hbase20:37039] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 22:00:01,913 DEBUG [RS:1;jenkins-hbase20:45945] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:01,913 INFO [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 22:00:01,913 INFO [RS:1;jenkins-hbase20:45945] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,45945,1676325598716; all regions closed. 2023-02-13 22:00:01,913 INFO [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,40899,1676325598751 2023-02-13 22:00:01,913 DEBUG [RS:2;jenkins-hbase20:40899] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3a542d03 to 127.0.0.1:53523 2023-02-13 22:00:01,913 INFO [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(1475): Waiting on 4 regions to close 2023-02-13 22:00:01,914 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 22:00:01,914 DEBUG [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(1479): Online Regions={78b5804c7ea67fd4601a55f3e6591a8d=restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d., f3962bf3439d5c7af4e285d7c3a11f11=restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11., 1588230740=hbase:meta,,1.1588230740, a9372b7742e56bdddc1c8be728e71577=restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577.} 2023-02-13 22:00:01,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing f02c6dea934906492e781696bf263bb2, disabling compactions & flushes 2023-02-13 22:00:01,915 DEBUG [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(1505): Waiting on 1588230740, 78b5804c7ea67fd4601a55f3e6591a8d, a9372b7742e56bdddc1c8be728e71577, f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:01,913 DEBUG [RS:2;jenkins-hbase20:40899] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:01,915 INFO [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 22:00:01,915 DEBUG [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer(1479): Online Regions={f02c6dea934906492e781696bf263bb2=hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2.} 2023-02-13 22:00:01,915 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 22:00:01,914 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 22:00:01,915 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 22:00:01,915 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 22:00:01,915 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 22:00:01,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 22:00:01,915 DEBUG [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer(1505): Waiting on f02c6dea934906492e781696bf263bb2 2023-02-13 22:00:01,915 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=5.15 KB heapSize=9.42 KB 2023-02-13 22:00:01,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. after waiting 0 ms 2023-02-13 22:00:01,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 22:00:01,915 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing f02c6dea934906492e781696bf263bb2 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-13 22:00:01,920 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:00:01,920 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableOne/78b5804c7ea67fd4601a55f3e6591a8d/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 22:00:01,920 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:00:01,924 DEBUG [RS:1;jenkins-hbase20:45945] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs 2023-02-13 22:00:01,924 INFO [RS:1;jenkins-hbase20:45945] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C45945%2C1676325598716:(num 1676325599201) 2023-02-13 22:00:01,924 DEBUG [RS:1;jenkins-hbase20:45945] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:01,924 INFO [RS:1;jenkins-hbase20:45945] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:00:01,928 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:01,928 INFO [RS:1;jenkins-hbase20:45945] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 22:00:01,928 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 78b5804c7ea67fd4601a55f3e6591a8d: Waiting for close lock at 1676325601910Running coprocessor pre-close hooks at 1676325601910Disabling compacts and flushes for region at 1676325601910Disabling writes for close at 1676325601911 (+1 ms)Writing region close event to WAL at 1676325601911Running coprocessor post-close hooks at 1676325601923 (+12 ms)Closed at 1676325601928 (+5 ms) 2023-02-13 22:00:01,928 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 22:00:01,928 INFO [RS:1;jenkins-hbase20:45945] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 22:00:01,928 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:01,928 INFO [RS:1;jenkins-hbase20:45945] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 22:00:01,928 INFO [RS:1;jenkins-hbase20:45945] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 22:00:01,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing f3962bf3439d5c7af4e285d7c3a11f11, disabling compactions & flushes 2023-02-13 22:00:01,929 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:01,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:01,929 INFO [RS:1;jenkins-hbase20:45945] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:45945 2023-02-13 22:00:01,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. after waiting 0 ms 2023-02-13 22:00:01,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:01,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableThree/f3962bf3439d5c7af4e285d7c3a11f11/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 22:00:01,939 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:01,939 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for f3962bf3439d5c7af4e285d7c3a11f11: Waiting for close lock at 1676325601928Running coprocessor pre-close hooks at 1676325601928Disabling compacts and flushes for region at 1676325601928Disabling writes for close at 1676325601930 (+2 ms)Writing region close event to WAL at 1676325601931 (+1 ms)Running coprocessor post-close hooks at 1676325601939 (+8 ms)Closed at 1676325601939 2023-02-13 22:00:01,939 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:01,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing a9372b7742e56bdddc1c8be728e71577, disabling compactions & flushes 2023-02-13 22:00:01,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:01,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:01,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. after waiting 0 ms 2023-02-13 22:00:01,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:01,944 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2/.tmp/info/8b6e49bbc34d4eff8eb4b1d99a049554 2023-02-13 22:00:01,945 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.78 KB at sequenceid=24 (bloomFilter=false), to=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/.tmp/info/92e4a1e065ce47d1a2cf7a9c11497952 2023-02-13 22:00:01,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableTwo/a9372b7742e56bdddc1c8be728e71577/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 22:00:01,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:01,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for a9372b7742e56bdddc1c8be728e71577: Waiting for close lock at 1676325601939Running coprocessor pre-close hooks at 1676325601939Disabling compacts and flushes for region at 1676325601939Disabling writes for close at 1676325601941 (+2 ms)Writing region close event to WAL at 1676325601944 (+3 ms)Running coprocessor post-close hooks at 1676325601948 (+4 ms)Closed at 1676325601948 2023-02-13 22:00:01,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:01,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2/.tmp/info/8b6e49bbc34d4eff8eb4b1d99a049554 as hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2/info/8b6e49bbc34d4eff8eb4b1d99a049554 2023-02-13 22:00:01,960 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2/info/8b6e49bbc34d4eff8eb4b1d99a049554, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 22:00:01,961 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:00:01,973 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for f02c6dea934906492e781696bf263bb2 in 58ms, sequenceid=6, compaction requested=false 2023-02-13 22:00:01,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-13 22:00:01,982 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=380 B at sequenceid=24 (bloomFilter=false), to=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/.tmp/table/27486f707bfe4541a3e353eba35c71f4 2023-02-13 22:00:01,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-13 22:00:01,984 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 22:00:01,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for f02c6dea934906492e781696bf263bb2: Waiting for close lock at 1676325601913Running coprocessor pre-close hooks at 1676325601913Disabling compacts and flushes for region at 1676325601913Disabling writes for close at 1676325601915 (+2 ms)Obtaining lock to block concurrent updates at 1676325601915Preparing flush snapshotting stores in f02c6dea934906492e781696bf263bb2 at 1676325601915Finished memstore snapshotting hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676325601915Flushing stores of hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. at 1676325601917 (+2 ms)Flushing f02c6dea934906492e781696bf263bb2/info: creating writer at 1676325601918 (+1 ms)Flushing f02c6dea934906492e781696bf263bb2/info: appending metadata at 1676325601928 (+10 ms)Flushing f02c6dea934906492e781696bf263bb2/info: closing flushed file at 1676325601928Flushing f02c6dea934906492e781696bf263bb2/info: reopening flushed file at 1676325601953 (+25 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for f02c6dea934906492e781696bf263bb2 in 58ms, sequenceid=6, compaction requested=false at 1676325601973 (+20 ms)Writing region close event to WAL at 1676325601979 (+6 ms)Running coprocessor post-close hooks at 1676325601983 (+4 ms)Closed at 1676325601984 (+1 ms) 2023-02-13 22:00:01,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 22:00:01,989 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/.tmp/info/92e4a1e065ce47d1a2cf7a9c11497952 as hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/info/92e4a1e065ce47d1a2cf7a9c11497952 2023-02-13 22:00:01,996 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/info/92e4a1e065ce47d1a2cf7a9c11497952, entries=40, sequenceid=24, filesize=9.5 K 2023-02-13 22:00:01,997 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/.tmp/table/27486f707bfe4541a3e353eba35c71f4 as hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/table/27486f707bfe4541a3e353eba35c71f4 2023-02-13 22:00:02,003 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/table/27486f707bfe4541a3e353eba35c71f4, entries=8, sequenceid=24, filesize=5.0 K 2023-02-13 22:00:02,004 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~5.15 KB/5270, heapSize ~9.14 KB/9360, currentSize=0 B/0 for 1588230740 in 89ms, sequenceid=24, compaction requested=false 2023-02-13 22:00:02,009 DEBUG [M:0;jenkins-hbase20:39807] zookeeper.ZKUtil(398): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 22:00:02,009 WARN [M:0;jenkins-hbase20:39807] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 22:00:02,009 INFO [M:0;jenkins-hbase20:39807] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:02,010 INFO [M:0;jenkins-hbase20:39807] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 22:00:02,010 DEBUG [M:0;jenkins-hbase20:39807] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 22:00:02,010 INFO [M:0;jenkins-hbase20:39807] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:00:02,010 DEBUG [M:0;jenkins-hbase20:39807] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:00:02,011 DEBUG [M:0;jenkins-hbase20:39807] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 22:00:02,011 DEBUG [M:0;jenkins-hbase20:39807] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:00:02,011 INFO [M:0;jenkins-hbase20:39807] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=64.26 KB heapSize=76.20 KB 2023-02-13 22:00:02,012 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40899-0x1015dda772d0003, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 22:00:02,012 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:45945-0x1015dda772d0002, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325598716 2023-02-13 22:00:02,012 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325598716 2023-02-13 22:00:02,012 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:45945-0x1015dda772d0002, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:02,012 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x1015dda772d0003, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325598716 2023-02-13 22:00:02,012 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x1015dda772d0003, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:02,012 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:02,012 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 22:00:02,012 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:02,022 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/recovered.edits/27.seqid, newMaxSeqId=27, maxSeqId=1 2023-02-13 22:00:02,023 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 22:00:02,024 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 22:00:02,024 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676325601914Running coprocessor pre-close hooks at 1676325601914Disabling compacts and flushes for region at 1676325601914Disabling writes for close at 1676325601915 (+1 ms)Obtaining lock to block concurrent updates at 1676325601915Preparing flush snapshotting stores in 1588230740 at 1676325601915Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=5270, getHeapSize=9600, getOffHeapSize=0, getCellsCount=48 at 1676325601915Flushing stores of hbase:meta,,1.1588230740 at 1676325601917 (+2 ms)Flushing 1588230740/info: creating writer at 1676325601918 (+1 ms)Flushing 1588230740/info: appending metadata at 1676325601930 (+12 ms)Flushing 1588230740/info: closing flushed file at 1676325601930Flushing 1588230740/table: creating writer at 1676325601951 (+21 ms)Flushing 1588230740/table: appending metadata at 1676325601954 (+3 ms)Flushing 1588230740/table: closing flushed file at 1676325601954Flushing 1588230740/info: reopening flushed file at 1676325601990 (+36 ms)Flushing 1588230740/table: reopening flushed file at 1676325601997 (+7 ms)Finished flush of dataSize ~5.15 KB/5270, heapSize ~9.14 KB/9360, currentSize=0 B/0 for 1588230740 in 89ms, sequenceid=24, compaction requested=false at 1676325602004 (+7 ms)Writing region close event to WAL at 1676325602009 (+5 ms)Running coprocessor post-close hooks at 1676325602023 (+14 ms)Closed at 1676325602024 (+1 ms) 2023-02-13 22:00:02,024 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 22:00:02,033 INFO [M:0;jenkins-hbase20:39807] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=64.26 KB at sequenceid=141 (bloomFilter=true), to=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/487f0953eb644333a831a9b897a6ff84 2023-02-13 22:00:02,040 DEBUG [M:0;jenkins-hbase20:39807] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/487f0953eb644333a831a9b897a6ff84 as hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/487f0953eb644333a831a9b897a6ff84 2023-02-13 22:00:02,047 INFO [M:0;jenkins-hbase20:39807] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/487f0953eb644333a831a9b897a6ff84, entries=17, sequenceid=141, filesize=8.5 K 2023-02-13 22:00:02,048 INFO [M:0;jenkins-hbase20:39807] regionserver.HRegion(2947): Finished flush of dataSize ~64.26 KB/65799, heapSize ~76.19 KB/78016, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 37ms, sequenceid=141, compaction requested=false 2023-02-13 22:00:02,050 INFO [M:0;jenkins-hbase20:39807] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:00:02,050 DEBUG [M:0;jenkins-hbase20:39807] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676325602010Disabling compacts and flushes for region at 1676325602010Disabling writes for close at 1676325602011 (+1 ms)Obtaining lock to block concurrent updates at 1676325602011Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676325602011Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=65799, getHeapSize=78016, getOffHeapSize=0, getCellsCount=158 at 1676325602012 (+1 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676325602014 (+2 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676325602014Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676325602023 (+9 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676325602023Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676325602041 (+18 ms)Finished flush of dataSize ~64.26 KB/65799, heapSize ~76.19 KB/78016, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 37ms, sequenceid=141, compaction requested=false at 1676325602048 (+7 ms)Writing region close event to WAL at 1676325602050 (+2 ms)Closed at 1676325602050 2023-02-13 22:00:02,054 INFO [M:0;jenkins-hbase20:39807] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 22:00:02,054 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 22:00:02,055 INFO [M:0;jenkins-hbase20:39807] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:39807 2023-02-13 22:00:02,058 DEBUG [M:0;jenkins-hbase20:39807] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase20.apache.org,39807,1676325598619 already deleted, retry=false 2023-02-13 22:00:02,060 INFO [regionserver/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-13 22:00:02,060 INFO [regionserver/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-13 22:00:02,067 INFO [regionserver/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-13 22:00:02,067 INFO [regionserver/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-13 22:00:02,115 INFO [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,37039,1676325598677; all regions closed. 2023-02-13 22:00:02,115 INFO [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,40899,1676325598751; all regions closed. 2023-02-13 22:00:02,124 DEBUG [RS:2;jenkins-hbase20:40899] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs 2023-02-13 22:00:02,124 INFO [RS:2;jenkins-hbase20:40899] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C40899%2C1676325598751:(num 1676325599201) 2023-02-13 22:00:02,124 DEBUG [RS:2;jenkins-hbase20:40899] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:02,124 INFO [RS:2;jenkins-hbase20:40899] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:00:02,124 INFO [RS:2;jenkins-hbase20:40899] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 22:00:02,124 INFO [RS:2;jenkins-hbase20:40899] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 22:00:02,124 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 22:00:02,125 INFO [RS:2;jenkins-hbase20:40899] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 22:00:02,125 INFO [RS:2;jenkins-hbase20:40899] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 22:00:02,125 INFO [RS:2;jenkins-hbase20:40899] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:40899 2023-02-13 22:00:02,128 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x1015dda772d0003, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,40899,1676325598751 2023-02-13 22:00:02,128 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,40899,1676325598751 2023-02-13 22:00:02,129 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1ea87184 rejected from java.util.concurrent.ThreadPoolExecutor@1a97f689[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:02,130 DEBUG [RS:0;jenkins-hbase20:37039] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs 2023-02-13 22:00:02,130 INFO [RS:0;jenkins-hbase20:37039] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C37039%2C1676325598677.meta:.meta(num 1676325599344) 2023-02-13 22:00:02,138 DEBUG [RS:0;jenkins-hbase20:37039] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs 2023-02-13 22:00:02,138 INFO [RS:0;jenkins-hbase20:37039] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C37039%2C1676325598677:(num 1676325599200) 2023-02-13 22:00:02,138 DEBUG [RS:0;jenkins-hbase20:37039] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:02,138 INFO [RS:0;jenkins-hbase20:37039] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:00:02,139 INFO [RS:0;jenkins-hbase20:37039] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 22:00:02,139 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 22:00:02,140 INFO [RS:0;jenkins-hbase20:37039] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:37039 2023-02-13 22:00:02,141 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 22:00:02,141 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5d08bbf1 rejected from java.util.concurrent.ThreadPoolExecutor@59373488[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:02,212 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:02,212 INFO [M:0;jenkins-hbase20:39807] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,39807,1676325598619; zookeeper connection closed. 2023-02-13 22:00:02,212 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@59a12e0a rejected from java.util.concurrent.ThreadPoolExecutor@5d9ea1e6[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 27] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:02,213 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:39807-0x1015dda772d0000, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:02,213 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@794f6fce rejected from java.util.concurrent.ThreadPoolExecutor@5d9ea1e6[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 27] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:02,313 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:45945-0x1015dda772d0002, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:02,313 INFO [RS:1;jenkins-hbase20:45945] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,45945,1676325598716; zookeeper connection closed. 2023-02-13 22:00:02,313 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@eb66ae9 rejected from java.util.concurrent.ThreadPoolExecutor@a89ffe7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:02,313 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1a4c473f] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1a4c473f 2023-02-13 22:00:02,313 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:45945-0x1015dda772d0002, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:02,313 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@43d28098 rejected from java.util.concurrent.ThreadPoolExecutor@a89ffe7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:02,615 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:02,615 INFO [RS:0;jenkins-hbase20:37039] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,37039,1676325598677; zookeeper connection closed. 2023-02-13 22:00:02,615 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@35ac1d2b rejected from java.util.concurrent.ThreadPoolExecutor@59373488[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:02,616 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3acf2134] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3acf2134 2023-02-13 22:00:02,616 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d0001, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:02,616 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4d101d0c rejected from java.util.concurrent.ThreadPoolExecutor@59373488[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:02,716 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x1015dda772d0003, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:02,716 INFO [RS:2;jenkins-hbase20:40899] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,40899,1676325598751; zookeeper connection closed. 2023-02-13 22:00:02,716 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@33f9a5e1 rejected from java.util.concurrent.ThreadPoolExecutor@1a97f689[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:02,716 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@32fd2e95] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@32fd2e95 2023-02-13 22:00:02,716 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x1015dda772d0003, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:02,716 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4b0646ac rejected from java.util.concurrent.ThreadPoolExecutor@1a97f689[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:02,716 INFO [Listener at localhost.localdomain/39067] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-13 22:00:02,716 INFO [Listener at localhost.localdomain/39067] master.TestRetainAssignmentOnRestart(93): Sleeping a bit 2023-02-13 22:00:04,717 INFO [Listener at localhost.localdomain/39067] master.TestRetainAssignmentOnRestart(96): Starting cluster the second time with the same ports 2023-02-13 22:00:04,727 INFO [Listener at localhost.localdomain/39067] client.ConnectionUtils(127): master/jenkins-hbase20:0 server-side Connection retries=45 2023-02-13 22:00:04,727 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:04,728 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:04,728 INFO [Listener at localhost.localdomain/39067] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 22:00:04,728 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:04,728 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 22:00:04,728 INFO [Listener at localhost.localdomain/39067] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 22:00:04,729 INFO [Listener at localhost.localdomain/39067] ipc.NettyRpcServer(120): Bind to /148.251.75.209:46519 2023-02-13 22:00:04,729 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:04,730 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:04,731 INFO [Listener at localhost.localdomain/39067] zookeeper.RecoverableZooKeeper(93): Process identifier=master:46519 connecting to ZooKeeper ensemble=127.0.0.1:53523 2023-02-13 22:00:04,734 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:465190x0, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 22:00:04,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:46519-0x1015dda772d0009 connected 2023-02-13 22:00:04,738 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 22:00:04,738 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(162): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/running 2023-02-13 22:00:04,739 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 22:00:04,739 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46519 2023-02-13 22:00:04,739 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46519 2023-02-13 22:00:04,740 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46519 2023-02-13 22:00:04,740 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46519 2023-02-13 22:00:04,740 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46519 2023-02-13 22:00:04,740 INFO [Listener at localhost.localdomain/39067] master.HMaster(439): hbase.rootdir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2, hbase.cluster.distributed=false 2023-02-13 22:00:04,741 DEBUG [pool-505-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: INIT 2023-02-13 22:00:04,742 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase20.apache.org,46519,1676325604727 2023-02-13 22:00:04,752 INFO [Listener at localhost.localdomain/39067] client.ConnectionUtils(127): regionserver/jenkins-hbase20:37039 server-side Connection retries=45 2023-02-13 22:00:04,752 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:04,753 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:04,753 INFO [Listener at localhost.localdomain/39067] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 22:00:04,753 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:04,753 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 22:00:04,753 INFO [Listener at localhost.localdomain/39067] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 22:00:04,754 INFO [Listener at localhost.localdomain/39067] ipc.NettyRpcServer(120): Bind to /148.251.75.209:37039 2023-02-13 22:00:04,754 INFO [Listener at localhost.localdomain/39067] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 22:00:04,755 DEBUG [Listener at localhost.localdomain/39067] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 22:00:04,756 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:04,757 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:04,757 INFO [Listener at localhost.localdomain/39067] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37039 connecting to ZooKeeper ensemble=127.0.0.1:53523 2023-02-13 22:00:04,763 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 22:00:04,765 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:370390x0, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 22:00:04,765 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase20.apache.org,46519,1676325604727 2023-02-13 22:00:04,767 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): regionserver:370390x0, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 22:00:04,767 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:37039-0x1015dda772d000a connected 2023-02-13 22:00:04,768 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 22:00:04,768 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:00:04,768 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(162): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/running 2023-02-13 22:00:04,768 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 22:00:04,768 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 22:00:04,769 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 22:00:04,769 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase20.apache.org,46519,1676325604727 from backup master directory 2023-02-13 22:00:04,769 INFO [M:1;jenkins-hbase20:46519] regionserver.HRegionServer(952): ClusterId : 1de008e8-3020-43bd-a4f6-073a3ec5ca42 2023-02-13 22:00:04,770 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase20.apache.org,46519,1676325604727 2023-02-13 22:00:04,770 WARN [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 22:00:04,770 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 22:00:04,770 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase20.apache.org,46519,1676325604727 2023-02-13 22:00:04,771 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 22:00:04,772 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37039 2023-02-13 22:00:04,775 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37039 2023-02-13 22:00:04,780 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37039 2023-02-13 22:00:04,781 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37039 2023-02-13 22:00:04,782 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37039 2023-02-13 22:00:04,783 INFO [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(952): ClusterId : 1de008e8-3020-43bd-a4f6-073a3ec5ca42 2023-02-13 22:00:04,783 DEBUG [RS:3;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 22:00:04,785 DEBUG [RS:3;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 22:00:04,785 DEBUG [RS:3;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 22:00:04,787 DEBUG [RS:3;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 22:00:04,788 DEBUG [RS:3;jenkins-hbase20:37039] zookeeper.ReadOnlyZKClient(139): Connect 0x1b54a430 to 127.0.0.1:53523 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 22:00:04,796 INFO [master/jenkins-hbase20:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:04,799 DEBUG [RS:3;jenkins-hbase20:37039] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1593592a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 22:00:04,799 DEBUG [RS:3;jenkins-hbase20:37039] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@11accf40, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 22:00:04,807 DEBUG [RS:3;jenkins-hbase20:37039] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase20:37039 2023-02-13 22:00:04,807 INFO [RS:3;jenkins-hbase20:37039] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 22:00:04,807 INFO [RS:3;jenkins-hbase20:37039] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 22:00:04,807 DEBUG [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 22:00:04,807 INFO [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,46519,1676325604727 with isa=jenkins-hbase20.apache.org/148.251.75.209:37039, startcode=1676325604752 2023-02-13 22:00:04,808 DEBUG [RS:3;jenkins-hbase20:37039] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 22:00:04,811 INFO [RS-EventLoopGroup-13-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:48681, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.9 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 22:00:04,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46519] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 22:00:04,812 DEBUG [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 22:00:04,812 WARN [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-13 22:00:04,834 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x64b9dbb6 to 127.0.0.1:53523 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 22:00:04,837 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@114cca34, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 22:00:04,837 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 22:00:04,838 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 22:00:04,839 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 22:00:04,845 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/WALs/jenkins-hbase20.apache.org,39807,1676325598619 to hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/WALs/jenkins-hbase20.apache.org,39807,1676325598619-dead as it is dead 2023-02-13 22:00:04,846 INFO [master/jenkins-hbase20:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/WALs/jenkins-hbase20.apache.org,39807,1676325598619-dead/jenkins-hbase20.apache.org%2C39807%2C1676325598619.1676325598863 2023-02-13 22:00:04,846 INFO [master/jenkins-hbase20:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/WALs/jenkins-hbase20.apache.org,39807,1676325598619-dead/jenkins-hbase20.apache.org%2C39807%2C1676325598619.1676325598863 after 0ms 2023-02-13 22:00:04,847 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/WALs/jenkins-hbase20.apache.org,39807,1676325598619-dead/jenkins-hbase20.apache.org%2C39807%2C1676325598619.1676325598863 to hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase20.apache.org%2C39807%2C1676325598619.1676325598863 2023-02-13 22:00:04,847 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/WALs/jenkins-hbase20.apache.org,39807,1676325598619-dead 2023-02-13 22:00:04,847 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/WALs/jenkins-hbase20.apache.org,46519,1676325604727 2023-02-13 22:00:04,850 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C46519%2C1676325604727, suffix=, logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/WALs/jenkins-hbase20.apache.org,46519,1676325604727, archiveDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/oldWALs, maxLogs=10 2023-02-13 22:00:04,868 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK] 2023-02-13 22:00:04,868 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK] 2023-02-13 22:00:04,868 DEBUG [RS-EventLoopGroup-14-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK] 2023-02-13 22:00:04,871 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/WALs/jenkins-hbase20.apache.org,46519,1676325604727/jenkins-hbase20.apache.org%2C46519%2C1676325604727.1676325604850 2023-02-13 22:00:04,871 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK], DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK], DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK]] 2023-02-13 22:00:04,871 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:04,871 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:04,871 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 22:00:04,871 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 22:00:04,872 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 22:00:04,873 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 22:00:04,874 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 22:00:04,879 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/487f0953eb644333a831a9b897a6ff84 2023-02-13 22:00:04,879 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:04,880 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5178): Found 1 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-02-13 22:00:04,881 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5275): Replaying edits from hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase20.apache.org%2C39807%2C1676325598619.1676325598863 2023-02-13 22:00:04,889 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5463): Applied 0, skipped 160, firstSequenceIdInLog=3, maxSequenceIdInLog=143, path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase20.apache.org%2C39807%2C1676325598619.1676325598863 2023-02-13 22:00:04,890 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5085): Deleted recovered.edits file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase20.apache.org%2C39807%2C1676325598619.1676325598863 2023-02-13 22:00:04,894 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 22:00:04,897 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/143.seqid, newMaxSeqId=143, maxSeqId=1 2023-02-13 22:00:04,898 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=144; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72494823, jitterRate=0.08025704324245453}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 22:00:04,898 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676325604871Initializing all the Stores at 1676325604872 (+1 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325604872Cleaning up temporary data from old regions at 1676325604891 (+19 ms)Cleaning up detritus from prior splits at 1676325604891Region opened successfully at 1676325604898 (+7 ms) 2023-02-13 22:00:04,898 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 22:00:04,899 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-13 22:00:04,899 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-13 22:00:04,899 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 22:00:04,900 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 22:00:04,904 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 22:00:04,904 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 22:00:04,905 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 22:00:04,905 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 22:00:04,905 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=restartTableOne 2023-02-13 22:00:04,906 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; CreateTableProcedure table=restartTableTwo 2023-02-13 22:00:04,906 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; CreateTableProcedure table=restartTableThree 2023-02-13 22:00:04,906 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 22:00:04,906 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 22:00:04,907 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 22:00:04,908 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,37039,1676325598677, table=hbase:meta, region=1588230740 2023-02-13 22:00:04,909 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 3 possibly 'live' servers, and 0 'splitting'. 2023-02-13 22:00:04,910 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,37039,1676325598677 already deleted, retry=false 2023-02-13 22:00:04,910 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase20.apache.org,37039,1676325598677 on jenkins-hbase20.apache.org,46519,1676325604727 2023-02-13 22:00:04,911 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase20.apache.org,37039,1676325598677, splitWal=true, meta=true 2023-02-13 22:00:04,912 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1736): Scheduled ServerCrashProcedure pid=18 for jenkins-hbase20.apache.org,37039,1676325598677 (carryingMeta=true) jenkins-hbase20.apache.org,37039,1676325598677/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@298285ae[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-02-13 22:00:04,912 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,45945,1676325598716 already deleted, retry=false 2023-02-13 22:00:04,912 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase20.apache.org,45945,1676325598716 on jenkins-hbase20.apache.org,46519,1676325604727 2023-02-13 22:00:04,913 INFO [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,46519,1676325604727 with isa=jenkins-hbase20.apache.org/148.251.75.209:37039, startcode=1676325604752 2023-02-13 22:00:04,913 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase20.apache.org,45945,1676325598716, splitWal=true, meta=false 2023-02-13 22:00:04,913 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1736): Scheduled ServerCrashProcedure pid=19 for jenkins-hbase20.apache.org,45945,1676325598716 (carryingMeta=false) jenkins-hbase20.apache.org,45945,1676325598716/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@65e5e8b9[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-02-13 22:00:04,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46519] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 22:00:04,914 DEBUG [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 22:00:04,914 WARN [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 200 ms and then retrying. 2023-02-13 22:00:04,914 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,40899,1676325598751 already deleted, retry=false 2023-02-13 22:00:04,914 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase20.apache.org,40899,1676325598751 on jenkins-hbase20.apache.org,46519,1676325604727 2023-02-13 22:00:04,915 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=20, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase20.apache.org,40899,1676325598751, splitWal=true, meta=false 2023-02-13 22:00:04,915 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1736): Scheduled ServerCrashProcedure pid=20 for jenkins-hbase20.apache.org,40899,1676325598751 (carryingMeta=false) jenkins-hbase20.apache.org,40899,1676325598751/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@25b2b5be[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-02-13 22:00:04,923 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 22:00:04,924 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 22:00:04,924 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/balancer 2023-02-13 22:00:04,924 INFO [master/jenkins-hbase20:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 22:00:04,925 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 22:00:04,925 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 22:00:04,926 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 22:00:04,927 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 22:00:04,927 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase20.apache.org,46519,1676325604727, sessionid=0x1015dda772d0009, setting cluster-up flag (Was=true) 2023-02-13 22:00:04,933 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 22:00:04,934 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,46519,1676325604727 2023-02-13 22:00:04,938 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 22:00:04,939 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,46519,1676325604727 2023-02-13 22:00:04,940 WARN [master/jenkins-hbase20:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/.hbase-snapshot/.tmp 2023-02-13 22:00:04,941 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 22:00:04,941 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 22:00:04,941 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 22:00:04,941 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-13 22:00:04,941 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase20:0, corePoolSize=10, maxPoolSize=10 2023-02-13 22:00:04,941 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:04,941 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-13 22:00:04,941 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:04,948 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676325634948 2023-02-13 22:00:04,949 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 22:00:04,949 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase20.apache.org,45945,1676325598716; numProcessing=1 2023-02-13 22:00:04,949 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 22:00:04,949 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=19, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase20.apache.org,45945,1676325598716, splitWal=true, meta=false 2023-02-13 22:00:04,949 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 22:00:04,950 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 22:00:04,950 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 22:00:04,950 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 22:00:04,950 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:04,950 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase20.apache.org,40899,1676325598751; numProcessing=2 2023-02-13 22:00:04,950 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase20.apache.org,37039,1676325598677; numProcessing=3 2023-02-13 22:00:04,950 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=20, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase20.apache.org,40899,1676325598751, splitWal=true, meta=false 2023-02-13 22:00:04,950 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=18, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase20.apache.org,37039,1676325598677, splitWal=true, meta=true 2023-02-13 22:00:04,951 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 22:00:04,951 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 22:00:04,951 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 22:00:04,952 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase20.apache.org,37039,1676325598677, splitWal=true, meta=true, isMeta: true 2023-02-13 22:00:04,955 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 22:00:04,955 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 22:00:04,956 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676325604956,5,FailOnTimeoutGroup] 2023-02-13 22:00:04,956 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676325604956,5,FailOnTimeoutGroup] 2023-02-13 22:00:04,956 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:04,956 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 22:00:04,956 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:04,956 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:04,956 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676325604766, completionTime=-1 2023-02-13 22:00:04,956 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=3 server(s), max=3 server(s), timeout=4500ms, lastChange=0ms 2023-02-13 22:00:04,957 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,37039,1676325598677-splitting 2023-02-13 22:00:04,957 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,37039,1676325598677-splitting dir is empty, no logs to split. 2023-02-13 22:00:04,958 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase20.apache.org,37039,1676325598677 WAL count=0, meta=true 2023-02-13 22:00:04,960 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,37039,1676325598677-splitting dir is empty, no logs to split. 2023-02-13 22:00:04,960 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase20.apache.org,37039,1676325598677 WAL count=0, meta=true 2023-02-13 22:00:04,960 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase20.apache.org,37039,1676325598677 WAL splitting is done? wals=0, meta=true 2023-02-13 22:00:04,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 22:00:04,963 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 22:00:04,965 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=21, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=jenkins-hbase20.apache.org,37039,1676325598677; forceNewPlan=false, retain=false 2023-02-13 22:00:05,115 INFO [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,46519,1676325604727 with isa=jenkins-hbase20.apache.org/148.251.75.209:37039, startcode=1676325604752 2023-02-13 22:00:05,115 WARN [jenkins-hbase20:46519] assignment.AssignmentManager(2157): No servers available; cannot place 1 unassigned regions. 2023-02-13 22:00:05,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46519] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:05,117 DEBUG [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2 2023-02-13 22:00:05,117 DEBUG [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:34137 2023-02-13 22:00:05,117 DEBUG [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 22:00:05,118 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:05,118 DEBUG [RS:3;jenkins-hbase20:37039] zookeeper.ZKUtil(162): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:05,119 WARN [RS:3;jenkins-hbase20:37039] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 22:00:05,119 INFO [RS:3;jenkins-hbase20:37039] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 22:00:05,119 DEBUG [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:05,119 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,37039,1676325604752] 2023-02-13 22:00:05,124 DEBUG [RS:3;jenkins-hbase20:37039] zookeeper.ZKUtil(162): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:05,125 DEBUG [RS:3;jenkins-hbase20:37039] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 22:00:05,125 INFO [RS:3;jenkins-hbase20:37039] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 22:00:05,127 INFO [RS:3;jenkins-hbase20:37039] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 22:00:05,127 INFO [RS:3;jenkins-hbase20:37039] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 22:00:05,127 INFO [RS:3;jenkins-hbase20:37039] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,127 INFO [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 22:00:05,129 INFO [RS:3;jenkins-hbase20:37039] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,129 DEBUG [RS:3;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:37039, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,129 DEBUG [RS:3;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:37039, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,129 DEBUG [RS:3;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:37039, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,129 DEBUG [RS:3;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,130 DEBUG [RS:3;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:37039, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,130 DEBUG [RS:3;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:37039, corePoolSize=2, maxPoolSize=2 2023-02-13 22:00:05,130 DEBUG [RS:3;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:37039, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,130 DEBUG [RS:3;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:37039, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,130 DEBUG [RS:3;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:37039, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,130 DEBUG [RS:3;jenkins-hbase20:37039] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:37039, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,130 INFO [RS:3;jenkins-hbase20:37039] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,131 INFO [RS:3;jenkins-hbase20:37039] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,131 INFO [RS:3;jenkins-hbase20:37039] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,140 INFO [RS:3;jenkins-hbase20:37039] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 22:00:05,141 INFO [RS:3;jenkins-hbase20:37039] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,37039,1676325604752-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,151 INFO [RS:3;jenkins-hbase20:37039] regionserver.Replication(203): jenkins-hbase20.apache.org,37039,1676325604752 started 2023-02-13 22:00:05,152 INFO [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,37039,1676325604752, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:37039, sessionid=0x1015dda772d000a 2023-02-13 22:00:05,152 DEBUG [RS:3;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 22:00:05,152 DEBUG [RS:3;jenkins-hbase20:37039] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:05,152 DEBUG [RS:3;jenkins-hbase20:37039] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,37039,1676325604752' 2023-02-13 22:00:05,152 DEBUG [RS:3;jenkins-hbase20:37039] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:05,153 DEBUG [RS:3;jenkins-hbase20:37039] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:05,153 DEBUG [RS:3;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 22:00:05,153 DEBUG [RS:3;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 22:00:05,153 DEBUG [RS:3;jenkins-hbase20:37039] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:05,153 DEBUG [RS:3;jenkins-hbase20:37039] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,37039,1676325604752' 2023-02-13 22:00:05,153 DEBUG [RS:3;jenkins-hbase20:37039] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 22:00:05,154 DEBUG [RS:3;jenkins-hbase20:37039] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 22:00:05,154 DEBUG [RS:3;jenkins-hbase20:37039] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 22:00:05,154 INFO [RS:3;jenkins-hbase20:37039] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 22:00:05,154 INFO [RS:3;jenkins-hbase20:37039] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 22:00:05,157 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=1; waited=201ms, expecting min=3 server(s), max=3 server(s), timeout=4500ms, lastChange=0ms 2023-02-13 22:00:05,166 INFO [Listener at localhost.localdomain/39067] client.ConnectionUtils(127): regionserver/jenkins-hbase20:45945 server-side Connection retries=45 2023-02-13 22:00:05,166 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:05,166 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:05,166 INFO [Listener at localhost.localdomain/39067] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 22:00:05,166 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:05,167 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 22:00:05,167 INFO [Listener at localhost.localdomain/39067] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 22:00:05,167 INFO [Listener at localhost.localdomain/39067] ipc.NettyRpcServer(120): Bind to /148.251.75.209:45945 2023-02-13 22:00:05,168 INFO [Listener at localhost.localdomain/39067] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 22:00:05,169 DEBUG [Listener at localhost.localdomain/39067] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 22:00:05,169 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:05,170 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:05,171 INFO [Listener at localhost.localdomain/39067] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45945 connecting to ZooKeeper ensemble=127.0.0.1:53523 2023-02-13 22:00:05,174 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:459450x0, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 22:00:05,175 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(162): regionserver:459450x0, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 22:00:05,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:45945-0x1015dda772d000d connected 2023-02-13 22:00:05,177 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(162): regionserver:45945-0x1015dda772d000d, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/running 2023-02-13 22:00:05,177 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): regionserver:45945-0x1015dda772d000d, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 22:00:05,178 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45945 2023-02-13 22:00:05,178 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45945 2023-02-13 22:00:05,179 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45945 2023-02-13 22:00:05,182 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45945 2023-02-13 22:00:05,182 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45945 2023-02-13 22:00:05,185 INFO [RS:4;jenkins-hbase20:45945] regionserver.HRegionServer(952): ClusterId : 1de008e8-3020-43bd-a4f6-073a3ec5ca42 2023-02-13 22:00:05,186 DEBUG [RS:4;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 22:00:05,191 DEBUG [RS:4;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 22:00:05,191 DEBUG [RS:4;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 22:00:05,193 DEBUG [RS:4;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 22:00:05,194 DEBUG [RS:4;jenkins-hbase20:45945] zookeeper.ReadOnlyZKClient(139): Connect 0x34b503fa to 127.0.0.1:53523 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 22:00:05,208 DEBUG [RS:4;jenkins-hbase20:45945] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3a72fb1c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 22:00:05,208 DEBUG [RS:4;jenkins-hbase20:45945] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3e4d688a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 22:00:05,215 DEBUG [RS:4;jenkins-hbase20:45945] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:4;jenkins-hbase20:45945 2023-02-13 22:00:05,215 INFO [RS:4;jenkins-hbase20:45945] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 22:00:05,215 INFO [RS:4;jenkins-hbase20:45945] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 22:00:05,215 DEBUG [RS:4;jenkins-hbase20:45945] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 22:00:05,216 INFO [RS:4;jenkins-hbase20:45945] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,46519,1676325604727 with isa=jenkins-hbase20.apache.org/148.251.75.209:45945, startcode=1676325605165 2023-02-13 22:00:05,216 DEBUG [RS:4;jenkins-hbase20:45945] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 22:00:05,218 INFO [RS-EventLoopGroup-13-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:43691, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.10 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 22:00:05,219 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46519] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,45945,1676325605165 2023-02-13 22:00:05,219 DEBUG [RS:4;jenkins-hbase20:45945] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2 2023-02-13 22:00:05,219 DEBUG [RS:4;jenkins-hbase20:45945] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:34137 2023-02-13 22:00:05,219 DEBUG [RS:4;jenkins-hbase20:45945] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 22:00:05,220 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:05,220 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:05,221 DEBUG [RS:4;jenkins-hbase20:45945] zookeeper.ZKUtil(162): regionserver:45945-0x1015dda772d000d, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325605165 2023-02-13 22:00:05,221 WARN [RS:4;jenkins-hbase20:45945] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 22:00:05,221 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,45945,1676325605165] 2023-02-13 22:00:05,221 INFO [RS:4;jenkins-hbase20:45945] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 22:00:05,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325605165 2023-02-13 22:00:05,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:05,221 DEBUG [RS:4;jenkins-hbase20:45945] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,45945,1676325605165 2023-02-13 22:00:05,227 DEBUG [RS:4;jenkins-hbase20:45945] zookeeper.ZKUtil(162): regionserver:45945-0x1015dda772d000d, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325605165 2023-02-13 22:00:05,227 DEBUG [RS:4;jenkins-hbase20:45945] zookeeper.ZKUtil(162): regionserver:45945-0x1015dda772d000d, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:05,228 DEBUG [RS:4;jenkins-hbase20:45945] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 22:00:05,228 INFO [RS:4;jenkins-hbase20:45945] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 22:00:05,231 INFO [RS:4;jenkins-hbase20:45945] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 22:00:05,231 INFO [RS:4;jenkins-hbase20:45945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 22:00:05,231 INFO [RS:4;jenkins-hbase20:45945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,231 INFO [RS:4;jenkins-hbase20:45945] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 22:00:05,233 INFO [RS:4;jenkins-hbase20:45945] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,233 DEBUG [RS:4;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:45945, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,234 DEBUG [RS:4;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:45945, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,234 DEBUG [RS:4;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:45945, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,234 DEBUG [RS:4;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:45945, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,234 DEBUG [RS:4;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:45945, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,234 DEBUG [RS:4;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:45945, corePoolSize=2, maxPoolSize=2 2023-02-13 22:00:05,234 DEBUG [RS:4;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:45945, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,234 DEBUG [RS:4;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:45945, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,234 DEBUG [RS:4;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:45945, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,234 DEBUG [RS:4;jenkins-hbase20:45945] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:45945, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,239 INFO [RS:4;jenkins-hbase20:45945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,239 INFO [RS:4;jenkins-hbase20:45945] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,239 INFO [RS:4;jenkins-hbase20:45945] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,249 INFO [RS:4;jenkins-hbase20:45945] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 22:00:05,249 INFO [RS:4;jenkins-hbase20:45945] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,45945,1676325605165-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,257 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=2; waited=301ms, expecting min=3 server(s), max=3 server(s), timeout=4500ms, lastChange=0ms 2023-02-13 22:00:05,257 INFO [RS:3;jenkins-hbase20:37039] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C37039%2C1676325604752, suffix=, logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,37039,1676325604752, archiveDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs, maxLogs=32 2023-02-13 22:00:05,263 INFO [RS:4;jenkins-hbase20:45945] regionserver.Replication(203): jenkins-hbase20.apache.org,45945,1676325605165 started 2023-02-13 22:00:05,263 INFO [RS:4;jenkins-hbase20:45945] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,45945,1676325605165, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:45945, sessionid=0x1015dda772d000d 2023-02-13 22:00:05,263 DEBUG [RS:4;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 22:00:05,263 DEBUG [RS:4;jenkins-hbase20:45945] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,45945,1676325605165 2023-02-13 22:00:05,263 DEBUG [RS:4;jenkins-hbase20:45945] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,45945,1676325605165' 2023-02-13 22:00:05,263 DEBUG [RS:4;jenkins-hbase20:45945] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:05,264 DEBUG [RS:4;jenkins-hbase20:45945] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:05,270 DEBUG [RS:4;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 22:00:05,270 DEBUG [RS:4;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 22:00:05,270 DEBUG [RS:4;jenkins-hbase20:45945] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,45945,1676325605165 2023-02-13 22:00:05,270 DEBUG [RS:4;jenkins-hbase20:45945] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,45945,1676325605165' 2023-02-13 22:00:05,270 DEBUG [RS:4;jenkins-hbase20:45945] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 22:00:05,270 DEBUG [RS:4;jenkins-hbase20:45945] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 22:00:05,272 DEBUG [RS:4;jenkins-hbase20:45945] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 22:00:05,272 INFO [RS:4;jenkins-hbase20:45945] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 22:00:05,272 INFO [RS:4;jenkins-hbase20:45945] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 22:00:05,275 DEBUG [RS-EventLoopGroup-14-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK] 2023-02-13 22:00:05,275 DEBUG [RS-EventLoopGroup-14-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK] 2023-02-13 22:00:05,275 DEBUG [RS-EventLoopGroup-14-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK] 2023-02-13 22:00:05,280 INFO [RS:3;jenkins-hbase20:37039] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,37039,1676325604752/jenkins-hbase20.apache.org%2C37039%2C1676325604752.1676325605258 2023-02-13 22:00:05,280 DEBUG [RS:3;jenkins-hbase20:37039] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK], DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK], DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK]] 2023-02-13 22:00:05,280 INFO [Listener at localhost.localdomain/39067] client.ConnectionUtils(127): regionserver/jenkins-hbase20:40899 server-side Connection retries=45 2023-02-13 22:00:05,280 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:05,281 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:05,281 INFO [Listener at localhost.localdomain/39067] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 22:00:05,281 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 22:00:05,281 INFO [Listener at localhost.localdomain/39067] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 22:00:05,281 INFO [Listener at localhost.localdomain/39067] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 22:00:05,287 INFO [Listener at localhost.localdomain/39067] ipc.NettyRpcServer(120): Bind to /148.251.75.209:40899 2023-02-13 22:00:05,288 INFO [Listener at localhost.localdomain/39067] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 22:00:05,293 DEBUG [Listener at localhost.localdomain/39067] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 22:00:05,293 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:05,295 INFO [Listener at localhost.localdomain/39067] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 22:00:05,296 INFO [Listener at localhost.localdomain/39067] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40899 connecting to ZooKeeper ensemble=127.0.0.1:53523 2023-02-13 22:00:05,300 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:408990x0, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 22:00:05,302 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(162): regionserver:408990x0, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 22:00:05,303 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(162): regionserver:408990x0, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/running 2023-02-13 22:00:05,304 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ZKUtil(164): regionserver:408990x0, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 22:00:05,311 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:40899-0x1015dda772d000f connected 2023-02-13 22:00:05,312 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40899 2023-02-13 22:00:05,315 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40899 2023-02-13 22:00:05,319 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40899 2023-02-13 22:00:05,326 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40899 2023-02-13 22:00:05,327 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40899 2023-02-13 22:00:05,329 INFO [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer(952): ClusterId : 1de008e8-3020-43bd-a4f6-073a3ec5ca42 2023-02-13 22:00:05,330 DEBUG [RS:5;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 22:00:05,332 DEBUG [RS:5;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 22:00:05,332 DEBUG [RS:5;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 22:00:05,334 DEBUG [RS:5;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 22:00:05,339 DEBUG [RS:5;jenkins-hbase20:40899] zookeeper.ReadOnlyZKClient(139): Connect 0x0e8a50b1 to 127.0.0.1:53523 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 22:00:05,351 DEBUG [RS:5;jenkins-hbase20:40899] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4a585bb8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 22:00:05,351 DEBUG [RS:5;jenkins-hbase20:40899] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3f4e3b73, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 22:00:05,362 DEBUG [RS:5;jenkins-hbase20:40899] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:5;jenkins-hbase20:40899 2023-02-13 22:00:05,362 INFO [RS:5;jenkins-hbase20:40899] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 22:00:05,362 INFO [RS:5;jenkins-hbase20:40899] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 22:00:05,362 DEBUG [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 22:00:05,363 INFO [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,46519,1676325604727 with isa=jenkins-hbase20.apache.org/148.251.75.209:40899, startcode=1676325605280 2023-02-13 22:00:05,363 DEBUG [RS:5;jenkins-hbase20:40899] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 22:00:05,365 INFO [jenkins-hbase20:46519] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 22:00:05,367 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,37039,1676325604752, state=OPENING 2023-02-13 22:00:05,369 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 22:00:05,369 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 22:00:05,369 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,37039,1676325604752}] 2023-02-13 22:00:05,373 INFO [RS-EventLoopGroup-13-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:41211, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.11 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 22:00:05,374 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46519] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,40899,1676325605280 2023-02-13 22:00:05,375 DEBUG [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2 2023-02-13 22:00:05,375 DEBUG [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:34137 2023-02-13 22:00:05,375 DEBUG [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 22:00:05,380 INFO [RS:4;jenkins-hbase20:45945] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C45945%2C1676325605165, suffix=, logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,45945,1676325605165, archiveDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs, maxLogs=32 2023-02-13 22:00:05,382 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:05,383 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:05,387 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:45945-0x1015dda772d000d, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:05,387 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,40899,1676325605280] 2023-02-13 22:00:05,387 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325605165 2023-02-13 22:00:05,387 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45945-0x1015dda772d000d, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325605165 2023-02-13 22:00:05,388 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,40899,1676325605280 2023-02-13 22:00:05,388 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45945-0x1015dda772d000d, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,40899,1676325605280 2023-02-13 22:00:05,388 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:45945-0x1015dda772d000d, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:05,388 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:05,391 DEBUG [RS:5;jenkins-hbase20:40899] zookeeper.ZKUtil(162): regionserver:40899-0x1015dda772d000f, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,40899,1676325605280 2023-02-13 22:00:05,391 WARN [RS:5;jenkins-hbase20:40899] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 22:00:05,391 INFO [RS:5;jenkins-hbase20:40899] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 22:00:05,391 DEBUG [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,40899,1676325605280 2023-02-13 22:00:05,401 DEBUG [RS:5;jenkins-hbase20:40899] zookeeper.ZKUtil(162): regionserver:40899-0x1015dda772d000f, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325605165 2023-02-13 22:00:05,401 DEBUG [RS:5;jenkins-hbase20:40899] zookeeper.ZKUtil(162): regionserver:40899-0x1015dda772d000f, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,40899,1676325605280 2023-02-13 22:00:05,402 DEBUG [RS:5;jenkins-hbase20:40899] zookeeper.ZKUtil(162): regionserver:40899-0x1015dda772d000f, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:05,403 DEBUG [RS:5;jenkins-hbase20:40899] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 22:00:05,403 INFO [RS:5;jenkins-hbase20:40899] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 22:00:05,407 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=451ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-13 22:00:05,407 WARN [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1230): hbase:meta,,1.1588230740 is NOT online; state={1588230740 state=OPENING, ts=1676325605369, server=jenkins-hbase20.apache.org,37039,1676325604752}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-02-13 22:00:05,412 DEBUG [RS-EventLoopGroup-15-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK] 2023-02-13 22:00:05,412 DEBUG [RS-EventLoopGroup-15-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK] 2023-02-13 22:00:05,415 INFO [RS:5;jenkins-hbase20:40899] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 22:00:05,419 DEBUG [RS-EventLoopGroup-15-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK] 2023-02-13 22:00:05,420 INFO [RS:5;jenkins-hbase20:40899] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 22:00:05,420 INFO [RS:5;jenkins-hbase20:40899] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,423 INFO [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 22:00:05,423 INFO [RS:4;jenkins-hbase20:45945] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,45945,1676325605165/jenkins-hbase20.apache.org%2C45945%2C1676325605165.1676325605380 2023-02-13 22:00:05,424 DEBUG [RS:4;jenkins-hbase20:45945] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK], DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK], DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK]] 2023-02-13 22:00:05,424 INFO [RS:5;jenkins-hbase20:40899] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,424 DEBUG [RS:5;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:40899, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,425 DEBUG [RS:5;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:40899, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,426 DEBUG [RS:5;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:40899, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,426 DEBUG [RS:5;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:40899, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,426 DEBUG [RS:5;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:40899, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,426 DEBUG [RS:5;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:40899, corePoolSize=2, maxPoolSize=2 2023-02-13 22:00:05,426 DEBUG [RS:5;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:40899, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,426 DEBUG [RS:5;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:40899, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,426 DEBUG [RS:5;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:40899, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,426 DEBUG [RS:5;jenkins-hbase20:40899] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:40899, corePoolSize=1, maxPoolSize=1 2023-02-13 22:00:05,432 INFO [RS:5;jenkins-hbase20:40899] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,432 INFO [RS:5;jenkins-hbase20:40899] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,432 INFO [RS:5;jenkins-hbase20:40899] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,446 INFO [RS:5;jenkins-hbase20:40899] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 22:00:05,446 INFO [RS:5;jenkins-hbase20:40899] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,40899,1676325605280-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:05,461 INFO [RS:5;jenkins-hbase20:40899] regionserver.Replication(203): jenkins-hbase20.apache.org,40899,1676325605280 started 2023-02-13 22:00:05,461 INFO [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,40899,1676325605280, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:40899, sessionid=0x1015dda772d000f 2023-02-13 22:00:05,461 DEBUG [RS:5;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 22:00:05,461 DEBUG [RS:5;jenkins-hbase20:40899] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,40899,1676325605280 2023-02-13 22:00:05,461 DEBUG [RS:5;jenkins-hbase20:40899] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,40899,1676325605280' 2023-02-13 22:00:05,461 INFO [Listener at localhost.localdomain/39067] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:05,461 DEBUG [RS:5;jenkins-hbase20:40899] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 22:00:05,462 DEBUG [RS:5;jenkins-hbase20:40899] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 22:00:05,462 DEBUG [RS:5;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 22:00:05,462 DEBUG [RS:5;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 22:00:05,463 DEBUG [RS:5;jenkins-hbase20:40899] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,40899,1676325605280 2023-02-13 22:00:05,463 DEBUG [RS:5;jenkins-hbase20:40899] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,40899,1676325605280' 2023-02-13 22:00:05,463 DEBUG [RS:5;jenkins-hbase20:40899] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 22:00:05,463 INFO [RS-EventLoopGroup-14-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42252, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 22:00:05,463 DEBUG [RS:5;jenkins-hbase20:40899] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 22:00:05,463 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37039] ipc.CallRunner(144): callId: 43 service: ClientService methodName: Get size: 90 connection: 148.251.75.209:42252 deadline: 1676325665463, exception=org.apache.hadoop.hbase.NotServingRegionException: hbase:meta,,1 is not online on jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:05,463 DEBUG [RS:5;jenkins-hbase20:40899] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 22:00:05,463 INFO [RS:5;jenkins-hbase20:40899] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 22:00:05,464 INFO [RS:5;jenkins-hbase20:40899] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 22:00:05,537 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:05,537 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 22:00:05,539 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42254, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=AdminService 2023-02-13 22:00:05,543 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 22:00:05,543 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 22:00:05,545 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C37039%2C1676325604752.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,37039,1676325604752, archiveDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs, maxLogs=32 2023-02-13 22:00:05,561 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK] 2023-02-13 22:00:05,561 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK] 2023-02-13 22:00:05,562 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK] 2023-02-13 22:00:05,565 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,37039,1676325604752/jenkins-hbase20.apache.org%2C37039%2C1676325604752.meta.1676325605545.meta 2023-02-13 22:00:05,565 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK], DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK], DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK]] 2023-02-13 22:00:05,565 WARN [ReadOnlyZKClient-127.0.0.1:53523@0x3bdb158c] client.ZKConnectionRegistry(168): Meta region is in state OPENING 2023-02-13 22:00:05,565 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:05,566 INFO [RS:5;jenkins-hbase20:40899] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C40899%2C1676325605280, suffix=, logDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,40899,1676325605280, archiveDir=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs, maxLogs=32 2023-02-13 22:00:05,566 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 22:00:05,566 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 22:00:05,566 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 22:00:05,566 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 22:00:05,566 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:05,566 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37039] ipc.CallRunner(144): callId: 44 service: ClientService methodName: Get size: 90 connection: 148.251.75.209:42252 deadline: 1676325665566, exception=org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region hbase:meta,,1 is opening on jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:05,566 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 22:00:05,566 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 22:00:05,569 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 22:00:05,571 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/info 2023-02-13 22:00:05,571 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/info 2023-02-13 22:00:05,571 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 22:00:05,581 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/info/92e4a1e065ce47d1a2cf7a9c11497952 2023-02-13 22:00:05,582 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:05,582 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 22:00:05,585 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/rep_barrier 2023-02-13 22:00:05,585 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/rep_barrier 2023-02-13 22:00:05,586 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 22:00:05,586 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:05,586 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 22:00:05,587 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK] 2023-02-13 22:00:05,587 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK] 2023-02-13 22:00:05,587 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK] 2023-02-13 22:00:05,588 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/table 2023-02-13 22:00:05,588 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/table 2023-02-13 22:00:05,588 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 22:00:05,595 INFO [RS:5;jenkins-hbase20:40899] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,40899,1676325605280/jenkins-hbase20.apache.org%2C40899%2C1676325605280.1676325605566 2023-02-13 22:00:05,596 DEBUG [RS:5;jenkins-hbase20:40899] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37781,DS-86b3f14b-3bd4-4674-9b67-39d2ebec7720,DISK], DatanodeInfoWithStorage[127.0.0.1:37895,DS-3c7651be-1d01-4017-8e7b-be8384d93e79,DISK], DatanodeInfoWithStorage[127.0.0.1:35895,DS-06ea857d-9d89-4e5e-8d94-fde6b7b56812,DISK]] 2023-02-13 22:00:05,600 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/table/27486f707bfe4541a3e353eba35c71f4 2023-02-13 22:00:05,600 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:05,601 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740 2023-02-13 22:00:05,603 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740 2023-02-13 22:00:05,607 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 22:00:05,609 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 22:00:05,610 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=28; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66079941, jitterRate=-0.015332147479057312}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 22:00:05,610 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676325605566Writing region info on filesystem at 1676325605566Initializing all the Stores at 1676325605567 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325605567Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325605569 (+2 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325605569Cleaning up temporary data from old regions at 1676325605606 (+37 ms)Cleaning up detritus from prior splits at 1676325605607 (+1 ms)Running coprocessor post-open hooks at 1676325605610 (+3 ms)Region opened successfully at 1676325605610 2023-02-13 22:00:05,611 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=22, masterSystemTime=1676325605537 2023-02-13 22:00:05,615 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 22:00:05,616 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:37039-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 22:00:05,616 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,37039,1676325604752, state=OPEN 2023-02-13 22:00:05,617 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 22:00:05,618 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 22:00:05,620 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-13 22:00:05,620 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,37039,1676325604752 in 249 msec 2023-02-13 22:00:05,622 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=18 2023-02-13 22:00:05,622 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 659 msec 2023-02-13 22:00:05,785 WARN [RPCClient-NioEventLoopGroup-6-7] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase20.apache.org/148.251.75.209:39807 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: jenkins-hbase20.apache.org/148.251.75.209:39807 Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716) at org.apache.hbase.thirdparty.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:337) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 22:00:05,785 WARN [Listener at localhost.localdomain/39067] client.ConnectionImplementation(411): Checking master connection java.net.ConnectException: Call to address=jenkins-hbase20.apache.org/148.251.75.209:39807 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: jenkins-hbase20.apache.org/148.251.75.209:39807 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:186) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:386) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:417) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:413) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:99) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:195) at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$300(NettyRpcConnection.java:76) at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:296) at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:287) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:321) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:337) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: jenkins-hbase20.apache.org/148.251.75.209:39807 Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716) at org.apache.hbase.thirdparty.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:337) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 22:00:05,785 DEBUG [RPCClient-NioEventLoopGroup-6-7] ipc.FailedServers(52): Added failed server with address jenkins-hbase20.apache.org/148.251.75.209:39807 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: jenkins-hbase20.apache.org/148.251.75.209:39807 2023-02-13 22:00:05,786 DEBUG [Listener at localhost.localdomain/39067] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase20.apache.org/148.251.75.209:39807 this server is in the failed servers list 2023-02-13 22:00:05,787 WARN [Listener at localhost.localdomain/39067] client.ConnectionImplementation(411): Checking master connection org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=jenkins-hbase20.apache.org/148.251.75.209:39807 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: jenkins-hbase20.apache.org/148.251.75.209:39807 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:236) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:386) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:417) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:413) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:430) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:326) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$200(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:582) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java) at org.apache.hadoop.hbase.client.ConnectionImplementation$MasterServiceState.isMasterRunning(ConnectionImplementation.java:1174) at org.apache.hadoop.hbase.client.ConnectionImplementation.lambda$masterConnectionStateSupplier$1(ConnectionImplementation.java:404) at org.apache.hadoop.hbase.client.ConnectionImplementation.isKeepAliveMasterConnectedAndRunning(ConnectionImplementation.java:1945) at org.apache.hadoop.hbase.client.ConnectionImplementation.getKeepAliveMasterService(ConnectionImplementation.java:1383) at org.apache.hadoop.hbase.client.ConnectionImplementation.getMaster(ConnectionImplementation.java:1373) at org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:57) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:102) at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:2954) at org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:542) at org.apache.hadoop.hbase.client.HTable.getDescriptor(HTable.java:228) at org.apache.hadoop.hbase.HBaseTestingUtility$9.evaluate(HBaseTestingUtility.java:3983) at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:183) at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:134) at org.apache.hadoop.hbase.HBaseCommonTestingUtility.waitFor(HBaseCommonTestingUtility.java:234) at org.apache.hadoop.hbase.HBaseTestingUtility.waitTableAvailable(HBaseTestingUtility.java:3136) at org.apache.hadoop.hbase.HBaseTestingUtility.waitTableAvailable(HBaseTestingUtility.java:3121) at org.apache.hadoop.hbase.master.TestRetainAssignmentOnRestart.testRetainAssignmentOnClusterRestart(TestRetainAssignmentOnRestart.java:108) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: jenkins-hbase20.apache.org/148.251.75.209:39807 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:350) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:427) ... 46 more 2023-02-13 22:00:05,788 DEBUG [Listener at localhost.localdomain/39067] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 22:00:05,790 INFO [RS-EventLoopGroup-13-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42606, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 22:00:05,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46519] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.PleaseHoldException: Master is initializing at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2823) at org.apache.hadoop.hbase.master.MasterRpcServices.getTableDescriptors(MasterRpcServices.java:1073) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 22:00:05,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46519] ipc.CallRunner(144): callId: 50 service: MasterService methodName: GetTableDescriptors size: 61 connection: 148.251.75.209:42606 deadline: 1676325665791, exception=org.apache.hadoop.hbase.PleaseHoldException: Master is initializing 2023-02-13 22:00:05,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46519] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.PleaseHoldException: Master is initializing at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2823) at org.apache.hadoop.hbase.master.MasterRpcServices.getTableDescriptors(MasterRpcServices.java:1073) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 22:00:05,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46519] ipc.CallRunner(144): callId: 52 service: MasterService methodName: GetTableDescriptors size: 61 connection: 148.251.75.209:42606 deadline: 1676325665892, exception=org.apache.hadoop.hbase.PleaseHoldException: Master is initializing 2023-02-13 22:00:05,908 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-02-13 22:00:06,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46519] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.PleaseHoldException: Master is initializing at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2823) at org.apache.hadoop.hbase.master.MasterRpcServices.getTableDescriptors(MasterRpcServices.java:1073) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 22:00:06,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46519] ipc.CallRunner(144): callId: 54 service: MasterService methodName: GetTableDescriptors size: 61 connection: 148.251.75.209:42606 deadline: 1676325666095, exception=org.apache.hadoop.hbase.PleaseHoldException: Master is initializing 2023-02-13 22:00:06,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46519] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.PleaseHoldException: Master is initializing at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2823) at org.apache.hadoop.hbase.master.MasterRpcServices.getTableDescriptors(MasterRpcServices.java:1073) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 22:00:06,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46519] ipc.CallRunner(144): callId: 56 service: MasterService methodName: GetTableDescriptors size: 61 connection: 148.251.75.209:42606 deadline: 1676325666397, exception=org.apache.hadoop.hbase.PleaseHoldException: Master is initializing 2023-02-13 22:00:06,408 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 22:00:06,411 DEBUG [hconnection-0x6228702e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 22:00:06,413 INFO [RS-EventLoopGroup-14-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42270, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=ClientService 2023-02-13 22:00:06,415 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=f02c6dea934906492e781696bf263bb2, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,40899,1676325598751, regionLocation=jenkins-hbase20.apache.org,40899,1676325598751, openSeqNum=2 2023-02-13 22:00:06,416 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=78b5804c7ea67fd4601a55f3e6591a8d, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37039,1676325598677, regionLocation=jenkins-hbase20.apache.org,37039,1676325598677, openSeqNum=2 2023-02-13 22:00:06,416 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=f3962bf3439d5c7af4e285d7c3a11f11, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37039,1676325598677, regionLocation=jenkins-hbase20.apache.org,37039,1676325598677, openSeqNum=2 2023-02-13 22:00:06,416 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=a9372b7742e56bdddc1c8be728e71577, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37039,1676325598677, regionLocation=jenkins-hbase20.apache.org,37039,1676325598677, openSeqNum=2 2023-02-13 22:00:06,416 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 22:00:06,416 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676325666416 2023-02-13 22:00:06,416 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676325726416 2023-02-13 22:00:06,416 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 8 msec 2023-02-13 22:00:06,433 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase20.apache.org,37039,1676325598677 had 4 regions 2023-02-13 22:00:06,433 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase20.apache.org,40899,1676325598751 had 1 regions 2023-02-13 22:00:06,433 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,46519,1676325604727-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:06,433 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase20.apache.org,45945,1676325598716 had 0 regions 2023-02-13 22:00:06,433 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,46519,1676325604727-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:06,434 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,46519,1676325604727-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:06,434 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase20:46519, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:06,434 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 22:00:06,434 WARN [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1230): hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. is NOT online; state={f02c6dea934906492e781696bf263bb2 state=OPEN, ts=1676325606415, server=jenkins-hbase20.apache.org,40899,1676325598751}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-02-13 22:00:06,436 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=20, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase20.apache.org,40899,1676325598751, splitWal=true, meta=false, isMeta: false 2023-02-13 22:00:06,437 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=18, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase20.apache.org,37039,1676325598677, splitWal=true, meta=true, isMeta: false 2023-02-13 22:00:06,436 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=19, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase20.apache.org,45945,1676325598716, splitWal=true, meta=false, isMeta: false 2023-02-13 22:00:06,440 DEBUG [PEWorker-5] master.MasterWalManager(318): Renamed region directory: hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,40899,1676325598751-splitting 2023-02-13 22:00:06,440 WARN [master/jenkins-hbase20:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase20.apache.org,40899,1676325598751/hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2., unknown_server=jenkins-hbase20.apache.org,37039,1676325598677/restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d., unknown_server=jenkins-hbase20.apache.org,37039,1676325598677/restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11., unknown_server=jenkins-hbase20.apache.org,37039,1676325598677/restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:06,441 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,40899,1676325598751-splitting dir is empty, no logs to split. 2023-02-13 22:00:06,441 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase20.apache.org,40899,1676325598751 WAL count=0, meta=false 2023-02-13 22:00:06,441 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,37039,1676325598677-splitting dir is empty, no logs to split. 2023-02-13 22:00:06,441 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase20.apache.org,37039,1676325598677 WAL count=0, meta=false 2023-02-13 22:00:06,442 DEBUG [PEWorker-3] master.MasterWalManager(318): Renamed region directory: hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,45945,1676325598716-splitting 2023-02-13 22:00:06,443 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,45945,1676325598716-splitting dir is empty, no logs to split. 2023-02-13 22:00:06,443 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase20.apache.org,45945,1676325598716 WAL count=0, meta=false 2023-02-13 22:00:06,444 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,40899,1676325598751-splitting dir is empty, no logs to split. 2023-02-13 22:00:06,445 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase20.apache.org,40899,1676325598751 WAL count=0, meta=false 2023-02-13 22:00:06,445 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase20.apache.org,40899,1676325598751 WAL splitting is done? wals=0, meta=false 2023-02-13 22:00:06,446 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,37039,1676325598677-splitting dir is empty, no logs to split. 2023-02-13 22:00:06,446 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase20.apache.org,37039,1676325598677 WAL count=0, meta=false 2023-02-13 22:00:06,446 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase20.apache.org,37039,1676325598677 WAL splitting is done? wals=0, meta=false 2023-02-13 22:00:06,447 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,45945,1676325598716-splitting dir is empty, no logs to split. 2023-02-13 22:00:06,447 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase20.apache.org,45945,1676325598716 WAL count=0, meta=false 2023-02-13 22:00:06,447 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase20.apache.org,45945,1676325598716 WAL splitting is done? wals=0, meta=false 2023-02-13 22:00:06,448 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableTwo, region=a9372b7742e56bdddc1c8be728e71577, ASSIGN}, {pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableOne, region=78b5804c7ea67fd4601a55f3e6591a8d, ASSIGN}, {pid=25, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableThree, region=f3962bf3439d5c7af4e285d7c3a11f11, ASSIGN}] 2023-02-13 22:00:06,449 INFO [PEWorker-5] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase20.apache.org,40899,1676325598751 failed, ignore...File hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,40899,1676325598751-splitting does not exist. 2023-02-13 22:00:06,450 INFO [PEWorker-3] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase20.apache.org,45945,1676325598716 failed, ignore...File hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/WALs/jenkins-hbase20.apache.org,45945,1676325598716-splitting does not exist. 2023-02-13 22:00:06,450 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableTwo, region=a9372b7742e56bdddc1c8be728e71577, ASSIGN 2023-02-13 22:00:06,450 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableOne, region=78b5804c7ea67fd4601a55f3e6591a8d, ASSIGN 2023-02-13 22:00:06,450 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=25, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=restartTableThree, region=f3962bf3439d5c7af4e285d7c3a11f11, ASSIGN 2023-02-13 22:00:06,451 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=20, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f02c6dea934906492e781696bf263bb2, ASSIGN}] 2023-02-13 22:00:06,452 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(189): Setting lastHost as the region location jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 22:00:06,452 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(189): Setting lastHost as the region location jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 22:00:06,452 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=25, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=restartTableThree, region=f3962bf3439d5c7af4e285d7c3a11f11, ASSIGN; state=OPEN, location=jenkins-hbase20.apache.org,37039,1676325598677; forceNewPlan=false, retain=true 2023-02-13 22:00:06,452 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=restartTableOne, region=78b5804c7ea67fd4601a55f3e6591a8d, ASSIGN; state=OPEN, location=jenkins-hbase20.apache.org,37039,1676325598677; forceNewPlan=false, retain=true 2023-02-13 22:00:06,452 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(189): Setting lastHost as the region location jenkins-hbase20.apache.org,37039,1676325598677 2023-02-13 22:00:06,453 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=20, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f02c6dea934906492e781696bf263bb2, ASSIGN 2023-02-13 22:00:06,453 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=restartTableTwo, region=a9372b7742e56bdddc1c8be728e71577, ASSIGN; state=OPEN, location=jenkins-hbase20.apache.org,37039,1676325598677; forceNewPlan=false, retain=true 2023-02-13 22:00:06,453 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase20.apache.org,45945,1676325598716 after splitting done 2023-02-13 22:00:06,453 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase20.apache.org,45945,1676325598716 from processing; numProcessing=2 2023-02-13 22:00:06,454 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(189): Setting lastHost as the region location jenkins-hbase20.apache.org,40899,1676325598751 2023-02-13 22:00:06,454 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=20, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=f02c6dea934906492e781696bf263bb2, ASSIGN; state=OPEN, location=jenkins-hbase20.apache.org,40899,1676325598751; forceNewPlan=false, retain=true 2023-02-13 22:00:06,454 INFO [jenkins-hbase20:46519] balancer.BaseLoadBalancer(1545): Reassigned 4 regions. 4 retained the pre-restart assignment. 2023-02-13 22:00:06,458 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; ServerCrashProcedure jenkins-hbase20.apache.org,45945,1676325598716, splitWal=true, meta=false in 1.5410 sec 2023-02-13 22:00:06,458 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=25 updating hbase:meta row=f3962bf3439d5c7af4e285d7c3a11f11, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:06,458 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=f02c6dea934906492e781696bf263bb2, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,40899,1676325605280 2023-02-13 22:00:06,458 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=78b5804c7ea67fd4601a55f3e6591a8d, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:06,458 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11.","families":{"info":[{"qualifier":"regioninfo","vlen":51,"tag":[],"timestamp":"1676325606458"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325606458"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325606458"}]},"ts":"1676325606458"} 2023-02-13 22:00:06,458 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676325606458"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325606458"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325606458"}]},"ts":"1676325606458"} 2023-02-13 22:00:06,458 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=a9372b7742e56bdddc1c8be728e71577, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:06,458 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325606458"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325606458"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325606458"}]},"ts":"1676325606458"} 2023-02-13 22:00:06,459 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325606458"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676325606458"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676325606458"}]},"ts":"1676325606458"} 2023-02-13 22:00:06,461 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=25, state=RUNNABLE; OpenRegionProcedure f3962bf3439d5c7af4e285d7c3a11f11, server=jenkins-hbase20.apache.org,37039,1676325604752}] 2023-02-13 22:00:06,462 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=26, state=RUNNABLE; OpenRegionProcedure f02c6dea934906492e781696bf263bb2, server=jenkins-hbase20.apache.org,40899,1676325605280}] 2023-02-13 22:00:06,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=24, state=RUNNABLE; OpenRegionProcedure 78b5804c7ea67fd4601a55f3e6591a8d, server=jenkins-hbase20.apache.org,37039,1676325604752}] 2023-02-13 22:00:06,464 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=23, state=RUNNABLE; OpenRegionProcedure a9372b7742e56bdddc1c8be728e71577, server=jenkins-hbase20.apache.org,37039,1676325604752}] 2023-02-13 22:00:06,616 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,40899,1676325605280 2023-02-13 22:00:06,616 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 22:00:06,618 INFO [RS-EventLoopGroup-16-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:44150, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=AdminService 2023-02-13 22:00:06,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] handler.AssignRegionHandler(128): Open restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:06,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(7850): Opening region: {ENCODED => f3962bf3439d5c7af4e285d7c3a11f11, NAME => 'restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:06,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table restartTableThree f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:06,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(865): Instantiated restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:06,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(7890): checking encryption for f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:06,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(7893): checking classloading for f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:06,621 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:40899-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 22:00:06,622 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(7850): Opening region: {ENCODED => f02c6dea934906492e781696bf263bb2, NAME => 'hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:06,622 INFO [StoreOpener-f3962bf3439d5c7af4e285d7c3a11f11-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family of region f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:06,622 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace f02c6dea934906492e781696bf263bb2 2023-02-13 22:00:06,622 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:06,622 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(7890): checking encryption for f02c6dea934906492e781696bf263bb2 2023-02-13 22:00:06,622 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(7893): checking classloading for f02c6dea934906492e781696bf263bb2 2023-02-13 22:00:06,623 DEBUG [StoreOpener-f3962bf3439d5c7af4e285d7c3a11f11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableThree/f3962bf3439d5c7af4e285d7c3a11f11/family 2023-02-13 22:00:06,623 DEBUG [StoreOpener-f3962bf3439d5c7af4e285d7c3a11f11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableThree/f3962bf3439d5c7af4e285d7c3a11f11/family 2023-02-13 22:00:06,623 INFO [StoreOpener-f02c6dea934906492e781696bf263bb2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region f02c6dea934906492e781696bf263bb2 2023-02-13 22:00:06,624 INFO [StoreOpener-f3962bf3439d5c7af4e285d7c3a11f11-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f3962bf3439d5c7af4e285d7c3a11f11 columnFamilyName family 2023-02-13 22:00:06,624 INFO [StoreOpener-f3962bf3439d5c7af4e285d7c3a11f11-1] regionserver.HStore(310): Store=f3962bf3439d5c7af4e285d7c3a11f11/family, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:06,624 DEBUG [StoreOpener-f02c6dea934906492e781696bf263bb2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2/info 2023-02-13 22:00:06,624 DEBUG [StoreOpener-f02c6dea934906492e781696bf263bb2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2/info 2023-02-13 22:00:06,625 INFO [StoreOpener-f02c6dea934906492e781696bf263bb2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f02c6dea934906492e781696bf263bb2 columnFamilyName info 2023-02-13 22:00:06,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableThree/f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:06,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableThree/f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:06,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1054): writing seq id for f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:06,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1071): Opened f3962bf3439d5c7af4e285d7c3a11f11; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63498479, jitterRate=-0.05379892885684967}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 22:00:06,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(964): Region open journal for f3962bf3439d5c7af4e285d7c3a11f11: Running coprocessor pre-open hook at 1676325606620Writing region info on filesystem at 1676325606620Initializing all the Stores at 1676325606621 (+1 ms)Instantiating store for column family {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325606621Cleaning up temporary data from old regions at 1676325606630 (+9 ms)Cleaning up detritus from prior splits at 1676325606630Running coprocessor post-open hooks at 1676325606633 (+3 ms)Region opened successfully at 1676325606633 2023-02-13 22:00:06,634 DEBUG [StoreOpener-f02c6dea934906492e781696bf263bb2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2/info/8b6e49bbc34d4eff8eb4b1d99a049554 2023-02-13 22:00:06,634 INFO [StoreOpener-f02c6dea934906492e781696bf263bb2-1] regionserver.HStore(310): Store=f02c6dea934906492e781696bf263bb2/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:06,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegionServer(2335): Post open deploy tasks for restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11., pid=27, masterSystemTime=1676325606613 2023-02-13 22:00:06,635 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2 2023-02-13 22:00:06,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegionServer(2362): Finished post open deploy task for restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:06,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] handler.AssignRegionHandler(156): Opened restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:06,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] handler.AssignRegionHandler(128): Open restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:06,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(7850): Opening region: {ENCODED => 78b5804c7ea67fd4601a55f3e6591a8d, NAME => 'restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:06,638 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2 2023-02-13 22:00:06,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table restartTableOne 78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:06,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(865): Instantiated restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:06,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(7890): checking encryption for 78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:06,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(7893): checking classloading for 78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:06,638 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=25 updating hbase:meta row=f3962bf3439d5c7af4e285d7c3a11f11, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:06,639 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11.","families":{"info":[{"qualifier":"regioninfo","vlen":51,"tag":[],"timestamp":"1676325606638"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325606638"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325606638"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325606638"}]},"ts":"1676325606638"} 2023-02-13 22:00:06,640 INFO [StoreOpener-78b5804c7ea67fd4601a55f3e6591a8d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family of region 78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:06,641 DEBUG [StoreOpener-78b5804c7ea67fd4601a55f3e6591a8d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableOne/78b5804c7ea67fd4601a55f3e6591a8d/family 2023-02-13 22:00:06,641 DEBUG [StoreOpener-78b5804c7ea67fd4601a55f3e6591a8d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableOne/78b5804c7ea67fd4601a55f3e6591a8d/family 2023-02-13 22:00:06,642 INFO [StoreOpener-78b5804c7ea67fd4601a55f3e6591a8d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 78b5804c7ea67fd4601a55f3e6591a8d columnFamilyName family 2023-02-13 22:00:06,642 INFO [StoreOpener-78b5804c7ea67fd4601a55f3e6591a8d-1] regionserver.HStore(310): Store=78b5804c7ea67fd4601a55f3e6591a8d/family, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:06,643 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(1054): writing seq id for f02c6dea934906492e781696bf263bb2 2023-02-13 22:00:06,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableOne/78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:06,644 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(1071): Opened f02c6dea934906492e781696bf263bb2; next sequenceid=10; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72012966, jitterRate=0.07307681441307068}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 22:00:06,644 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(964): Region open journal for f02c6dea934906492e781696bf263bb2: Running coprocessor pre-open hook at 1676325606622Writing region info on filesystem at 1676325606622Initializing all the Stores at 1676325606623 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676325606623Cleaning up temporary data from old regions at 1676325606639 (+16 ms)Cleaning up detritus from prior splits at 1676325606640 (+1 ms)Running coprocessor post-open hooks at 1676325606644 (+4 ms)Region opened successfully at 1676325606644 2023-02-13 22:00:06,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableOne/78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:06,647 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2., pid=28, masterSystemTime=1676325606616 2023-02-13 22:00:06,650 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=25 2023-02-13 22:00:06,650 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=25, state=SUCCESS; OpenRegionProcedure f3962bf3439d5c7af4e285d7c3a11f11, server=jenkins-hbase20.apache.org,37039,1676325604752 in 180 msec 2023-02-13 22:00:06,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1054): writing seq id for 78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:06,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1071): Opened 78b5804c7ea67fd4601a55f3e6591a8d; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72580287, jitterRate=0.08153055608272552}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 22:00:06,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(964): Region open journal for 78b5804c7ea67fd4601a55f3e6591a8d: Running coprocessor pre-open hook at 1676325606638Writing region info on filesystem at 1676325606638Initializing all the Stores at 1676325606639 (+1 ms)Instantiating store for column family {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325606639Cleaning up temporary data from old regions at 1676325606645 (+6 ms)Cleaning up detritus from prior splits at 1676325606645Running coprocessor post-open hooks at 1676325606653 (+8 ms)Region opened successfully at 1676325606653 2023-02-13 22:00:06,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegionServer(2335): Post open deploy tasks for restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d., pid=29, masterSystemTime=1676325606613 2023-02-13 22:00:06,656 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=restartTableThree, region=f3962bf3439d5c7af4e285d7c3a11f11, ASSIGN in 202 msec 2023-02-13 22:00:06,656 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=f02c6dea934906492e781696bf263bb2, regionState=OPEN, openSeqNum=10, regionLocation=jenkins-hbase20.apache.org,40899,1676325605280 2023-02-13 22:00:06,656 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676325606656"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325606656"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325606656"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325606656"}]},"ts":"1676325606656"} 2023-02-13 22:00:06,657 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 22:00:06,658 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:40899-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 22:00:06,659 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegionServer(2362): Finished post open deploy task for restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:06,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] handler.AssignRegionHandler(156): Opened restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:06,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] handler.AssignRegionHandler(128): Open restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:06,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(7850): Opening region: {ENCODED => a9372b7742e56bdddc1c8be728e71577, NAME => 'restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577.', STARTKEY => '', ENDKEY => ''} 2023-02-13 22:00:06,660 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=78b5804c7ea67fd4601a55f3e6591a8d, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:06,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table restartTableTwo a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:06,660 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325606660"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325606660"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325606660"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325606660"}]},"ts":"1676325606660"} 2023-02-13 22:00:06,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(865): Instantiated restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 22:00:06,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(7890): checking encryption for a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:06,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(7893): checking classloading for a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:06,663 INFO [StoreOpener-a9372b7742e56bdddc1c8be728e71577-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family of region a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:06,663 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=26 2023-02-13 22:00:06,664 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=26, state=SUCCESS; OpenRegionProcedure f02c6dea934906492e781696bf263bb2, server=jenkins-hbase20.apache.org,40899,1676325605280 in 198 msec 2023-02-13 22:00:06,664 DEBUG [StoreOpener-a9372b7742e56bdddc1c8be728e71577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableTwo/a9372b7742e56bdddc1c8be728e71577/family 2023-02-13 22:00:06,664 DEBUG [StoreOpener-a9372b7742e56bdddc1c8be728e71577-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableTwo/a9372b7742e56bdddc1c8be728e71577/family 2023-02-13 22:00:06,665 INFO [StoreOpener-a9372b7742e56bdddc1c8be728e71577-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9372b7742e56bdddc1c8be728e71577 columnFamilyName family 2023-02-13 22:00:06,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=24 2023-02-13 22:00:06,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=24, state=SUCCESS; OpenRegionProcedure 78b5804c7ea67fd4601a55f3e6591a8d, server=jenkins-hbase20.apache.org,37039,1676325604752 in 199 msec 2023-02-13 22:00:06,666 INFO [StoreOpener-a9372b7742e56bdddc1c8be728e71577-1] regionserver.HStore(310): Store=a9372b7742e56bdddc1c8be728e71577/family, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 22:00:06,666 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=20 2023-02-13 22:00:06,666 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase20.apache.org,40899,1676325598751 after splitting done 2023-02-13 22:00:06,666 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=20, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=f02c6dea934906492e781696bf263bb2, ASSIGN in 213 msec 2023-02-13 22:00:06,667 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase20.apache.org,40899,1676325598751 from processing; numProcessing=1 2023-02-13 22:00:06,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableTwo/a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:06,668 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=restartTableOne, region=78b5804c7ea67fd4601a55f3e6591a8d, ASSIGN in 217 msec 2023-02-13 22:00:06,669 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=20, state=SUCCESS; ServerCrashProcedure jenkins-hbase20.apache.org,40899,1676325598751, splitWal=true, meta=false in 1.7530 sec 2023-02-13 22:00:06,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableTwo/a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:06,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1054): writing seq id for a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:06,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1071): Opened a9372b7742e56bdddc1c8be728e71577; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60606513, jitterRate=-0.0968925803899765}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 22:00:06,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(964): Region open journal for a9372b7742e56bdddc1c8be728e71577: Running coprocessor pre-open hook at 1676325606661Writing region info on filesystem at 1676325606661Initializing all the Stores at 1676325606662 (+1 ms)Instantiating store for column family {NAME => 'family', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676325606662Cleaning up temporary data from old regions at 1676325606669 (+7 ms)Cleaning up detritus from prior splits at 1676325606669Running coprocessor post-open hooks at 1676325606672 (+3 ms)Region opened successfully at 1676325606672 2023-02-13 22:00:06,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegionServer(2335): Post open deploy tasks for restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577., pid=30, masterSystemTime=1676325606613 2023-02-13 22:00:06,675 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegionServer(2362): Finished post open deploy task for restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:06,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:37039-0] handler.AssignRegionHandler(156): Opened restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:06,676 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=a9372b7742e56bdddc1c8be728e71577, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:06,676 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676325606675"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676325606675"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676325606675"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676325606675"}]},"ts":"1676325606675"} 2023-02-13 22:00:06,680 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=23 2023-02-13 22:00:06,680 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=23, state=SUCCESS; OpenRegionProcedure a9372b7742e56bdddc1c8be728e71577, server=jenkins-hbase20.apache.org,37039,1676325604752 in 214 msec 2023-02-13 22:00:06,683 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=18 2023-02-13 22:00:06,683 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase20.apache.org,37039,1676325598677 after splitting done 2023-02-13 22:00:06,683 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=restartTableTwo, region=a9372b7742e56bdddc1c8be728e71577, ASSIGN in 232 msec 2023-02-13 22:00:06,683 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase20.apache.org,37039,1676325598677 from processing; numProcessing=0 2023-02-13 22:00:06,685 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; ServerCrashProcedure jenkins-hbase20.apache.org,37039,1676325598677, splitWal=true, meta=true in 1.7730 sec 2023-02-13 22:00:06,801 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 22:00:06,801 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-13 22:00:06,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46519] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.PleaseHoldException: Master is initializing at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2823) at org.apache.hadoop.hbase.master.MasterRpcServices.getTableDescriptors(MasterRpcServices.java:1073) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 22:00:06,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46519] ipc.CallRunner(144): callId: 58 service: MasterService methodName: GetTableDescriptors size: 61 connection: 148.251.75.209:42606 deadline: 1676325666902, exception=org.apache.hadoop.hbase.PleaseHoldException: Master is initializing 2023-02-13 22:00:07,435 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on existing znode=/hbase/namespace 2023-02-13 22:00:07,439 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 22:00:07,441 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:44156, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=ClientService 2023-02-13 22:00:07,453 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 22:00:07,454 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 22:00:07,454 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.684sec 2023-02-13 22:00:07,455 INFO [master/jenkins-hbase20:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 22:00:07,455 INFO [master/jenkins-hbase20:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 22:00:07,455 INFO [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 22:00:07,455 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,46519,1676325604727-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 22:00:07,455 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,46519,1676325604727-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 22:00:07,459 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 22:00:07,909 INFO [Listener at localhost.localdomain/39067] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:07,919 INFO [Listener at localhost.localdomain/39067] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:07,927 INFO [Listener at localhost.localdomain/39067] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 22:00:07,928 INFO [Listener at localhost.localdomain/39067] master.SnapshotOfRegionAssignmentFromMeta(102): Start to scan the hbase:meta for the current region assignment snappshot 2023-02-13 22:00:07,932 INFO [Listener at localhost.localdomain/39067] master.SnapshotOfRegionAssignmentFromMeta(164): Finished to scan the hbase:meta for the current region assignmentsnapshot 2023-02-13 22:00:07,932 INFO [Listener at localhost.localdomain/39067] master.TestRetainAssignmentOnRestart(119): Key={ENCODED => a9372b7742e56bdddc1c8be728e71577, NAME => 'restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577.', STARTKEY => '', ENDKEY => ''} oldServer=jenkins-hbase20.apache.org,37039,1676325598677, currentServer=jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:07,933 INFO [Listener at localhost.localdomain/39067] master.TestRetainAssignmentOnRestart(119): Key={ENCODED => f02c6dea934906492e781696bf263bb2, NAME => 'hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2.', STARTKEY => '', ENDKEY => ''} oldServer=jenkins-hbase20.apache.org,40899,1676325598751, currentServer=jenkins-hbase20.apache.org,40899,1676325605280 2023-02-13 22:00:07,933 INFO [Listener at localhost.localdomain/39067] master.TestRetainAssignmentOnRestart(119): Key={ENCODED => 78b5804c7ea67fd4601a55f3e6591a8d, NAME => 'restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d.', STARTKEY => '', ENDKEY => ''} oldServer=jenkins-hbase20.apache.org,37039,1676325598677, currentServer=jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:07,933 INFO [Listener at localhost.localdomain/39067] master.TestRetainAssignmentOnRestart(119): Key={ENCODED => f3962bf3439d5c7af4e285d7c3a11f11, NAME => 'restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11.', STARTKEY => '', ENDKEY => ''} oldServer=jenkins-hbase20.apache.org,37039,1676325598677, currentServer=jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:07,933 INFO [Listener at localhost.localdomain/39067] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-13 22:00:07,933 INFO [Listener at localhost.localdomain/39067] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 22:00:07,933 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3bdb158c to 127.0.0.1:53523 2023-02-13 22:00:07,933 DEBUG [Listener at localhost.localdomain/39067] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:07,933 DEBUG [Listener at localhost.localdomain/39067] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-13 22:00:07,933 DEBUG [Listener at localhost.localdomain/39067] util.JVMClusterUtil(257): Found active master hash=1537100762, stopped=false 2023-02-13 22:00:07,934 INFO [Listener at localhost.localdomain/39067] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase20.apache.org,46519,1676325604727 2023-02-13 22:00:07,934 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x1015dda772d000f, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 22:00:07,934 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 22:00:07,934 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:45945-0x1015dda772d000d, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 22:00:07,934 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 22:00:07,934 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:00:07,934 INFO [Listener at localhost.localdomain/39067] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 22:00:07,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 22:00:07,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40899-0x1015dda772d000f, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 22:00:07,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45945-0x1015dda772d000d, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 22:00:07,936 DEBUG [Listener at localhost.localdomain/39067] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x64b9dbb6 to 127.0.0.1:53523 2023-02-13 22:00:07,937 DEBUG [Listener at localhost.localdomain/39067] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:07,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 22:00:07,937 INFO [Listener at localhost.localdomain/39067] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,37039,1676325604752' ***** 2023-02-13 22:00:07,937 INFO [Listener at localhost.localdomain/39067] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 22:00:07,937 INFO [Listener at localhost.localdomain/39067] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,45945,1676325605165' ***** 2023-02-13 22:00:07,937 INFO [RS:3;jenkins-hbase20:37039] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 22:00:07,937 INFO [RS:3;jenkins-hbase20:37039] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 22:00:07,937 INFO [Listener at localhost.localdomain/39067] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 22:00:07,937 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 22:00:07,937 INFO [RS:4;jenkins-hbase20:45945] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 22:00:07,937 INFO [Listener at localhost.localdomain/39067] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,40899,1676325605280' ***** 2023-02-13 22:00:07,937 INFO [RS:3;jenkins-hbase20:37039] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 22:00:07,938 INFO [Listener at localhost.localdomain/39067] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 22:00:07,938 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 22:00:07,938 INFO [RS:5;jenkins-hbase20:40899] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 22:00:07,938 INFO [RS:5;jenkins-hbase20:40899] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 22:00:07,938 INFO [RS:4;jenkins-hbase20:45945] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 22:00:07,939 INFO [RS:5;jenkins-hbase20:40899] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 22:00:07,938 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 22:00:07,939 INFO [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer(3304): Received CLOSE for f02c6dea934906492e781696bf263bb2 2023-02-13 22:00:07,938 INFO [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(3304): Received CLOSE for 78b5804c7ea67fd4601a55f3e6591a8d 2023-02-13 22:00:07,939 INFO [RS:4;jenkins-hbase20:45945] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 22:00:07,940 INFO [RS:4;jenkins-hbase20:45945] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,45945,1676325605165 2023-02-13 22:00:07,940 INFO [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,40899,1676325605280 2023-02-13 22:00:07,940 DEBUG [RS:4;jenkins-hbase20:45945] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x34b503fa to 127.0.0.1:53523 2023-02-13 22:00:07,940 INFO [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(3304): Received CLOSE for f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:07,940 DEBUG [RS:5;jenkins-hbase20:40899] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0e8a50b1 to 127.0.0.1:53523 2023-02-13 22:00:07,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(1603): Closing f02c6dea934906492e781696bf263bb2, disabling compactions & flushes 2023-02-13 22:00:07,941 DEBUG [RS:5;jenkins-hbase20:40899] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:07,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1603): Closing 78b5804c7ea67fd4601a55f3e6591a8d, disabling compactions & flushes 2023-02-13 22:00:07,941 INFO [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(3304): Received CLOSE for a9372b7742e56bdddc1c8be728e71577 2023-02-13 22:00:07,940 DEBUG [RS:4;jenkins-hbase20:45945] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:07,941 INFO [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:07,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1625): Closing region restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:07,941 INFO [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 22:00:07,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 22:00:07,941 DEBUG [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer(1479): Online Regions={f02c6dea934906492e781696bf263bb2=hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2.} 2023-02-13 22:00:07,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:07,941 DEBUG [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer(1505): Waiting on f02c6dea934906492e781696bf263bb2 2023-02-13 22:00:07,941 DEBUG [RS:3;jenkins-hbase20:37039] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1b54a430 to 127.0.0.1:53523 2023-02-13 22:00:07,941 INFO [RS:4;jenkins-hbase20:45945] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,45945,1676325605165; all regions closed. 2023-02-13 22:00:07,941 DEBUG [RS:3;jenkins-hbase20:37039] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:07,942 INFO [RS:3;jenkins-hbase20:37039] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 22:00:07,942 INFO [RS:3;jenkins-hbase20:37039] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 22:00:07,942 INFO [RS:3;jenkins-hbase20:37039] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 22:00:07,942 INFO [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 22:00:07,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1713): Acquired close lock on restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. after waiting 0 ms 2023-02-13 22:00:07,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 22:00:07,942 INFO [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(1475): Waiting on 4 regions to close 2023-02-13 22:00:07,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1723): Updates disabled for region restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:07,942 DEBUG [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(1479): Online Regions={78b5804c7ea67fd4601a55f3e6591a8d=restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d., f3962bf3439d5c7af4e285d7c3a11f11=restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11., 1588230740=hbase:meta,,1.1588230740, a9372b7742e56bdddc1c8be728e71577=restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577.} 2023-02-13 22:00:07,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. after waiting 0 ms 2023-02-13 22:00:07,943 DEBUG [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(1505): Waiting on 1588230740, 78b5804c7ea67fd4601a55f3e6591a8d, a9372b7742e56bdddc1c8be728e71577, f3962bf3439d5c7af4e285d7c3a11f11 2023-02-13 22:00:07,943 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 22:00:07,943 INFO [regionserver/jenkins-hbase20:45945.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:00:07,943 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 22:00:07,944 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 22:00:07,944 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 22:00:07,944 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 22:00:07,944 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=3.82 KB heapSize=6.91 KB 2023-02-13 22:00:07,943 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 22:00:07,953 DEBUG [RS:4;jenkins-hbase20:45945] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs 2023-02-13 22:00:07,953 INFO [RS:4;jenkins-hbase20:45945] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C45945%2C1676325605165:(num 1676325605380) 2023-02-13 22:00:07,953 DEBUG [RS:4;jenkins-hbase20:45945] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:07,953 INFO [RS:4;jenkins-hbase20:45945] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:00:07,953 INFO [RS:4;jenkins-hbase20:45945] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:45945 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 22:00:07,954 INFO [RS:4;jenkins-hbase20:45945] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 22:00:07,954 INFO [regionserver/jenkins-hbase20:45945.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 22:00:07,954 INFO [RS:4;jenkins-hbase20:45945] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 22:00:07,954 INFO [RS:4;jenkins-hbase20:45945] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 22:00:07,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableOne/78b5804c7ea67fd4601a55f3e6591a8d/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 22:00:07,955 INFO [RS:4;jenkins-hbase20:45945] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:45945 2023-02-13 22:00:07,956 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1837): Closed restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:07,956 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1557): Region close journal for 78b5804c7ea67fd4601a55f3e6591a8d: Waiting for close lock at 1676325607940Running coprocessor pre-close hooks at 1676325607940Disabling compacts and flushes for region at 1676325607940Disabling writes for close at 1676325607942 (+2 ms)Writing region close event to WAL at 1676325607943 (+1 ms)Running coprocessor post-close hooks at 1676325607956 (+13 ms)Closed at 1676325607956 2023-02-13 22:00:07,956 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] handler.CloseRegionHandler(117): Closed restartTableOne,,1676325600039.78b5804c7ea67fd4601a55f3e6591a8d. 2023-02-13 22:00:07,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1603): Closing f3962bf3439d5c7af4e285d7c3a11f11, disabling compactions & flushes 2023-02-13 22:00:07,958 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1625): Closing region restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:07,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:07,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1713): Acquired close lock on restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. after waiting 0 ms 2023-02-13 22:00:07,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1723): Updates disabled for region restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:07,958 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:07,959 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x1015dda772d000f, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325605165 2023-02-13 22:00:07,959 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x1015dda772d000f, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:07,959 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325605165 2023-02-13 22:00:07,959 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:07,959 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:45945-0x1015dda772d000d, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,45945,1676325605165 2023-02-13 22:00:07,959 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@333b8bae rejected from java.util.concurrent.ThreadPoolExecutor@37041911[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:07,959 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:45945-0x1015dda772d000d, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:07,959 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2cb78ace rejected from java.util.concurrent.ThreadPoolExecutor@37041911[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:07,959 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,45945,1676325605165] 2023-02-13 22:00:07,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:40899-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/namespace/f02c6dea934906492e781696bf263bb2/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=9 2023-02-13 22:00:07,960 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,45945,1676325605165; numProcessing=1 2023-02-13 22:00:07,960 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,45945,1676325605165 already deleted, retry=false 2023-02-13 22:00:07,960 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 22:00:07,960 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,45945,1676325605165 expired; onlineServers=2 2023-02-13 22:00:07,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:40899-0] regionserver.HRegion(1557): Region close journal for f02c6dea934906492e781696bf263bb2: Waiting for close lock at 1676325607940Running coprocessor pre-close hooks at 1676325607940Disabling compacts and flushes for region at 1676325607940Disabling writes for close at 1676325607943 (+3 ms)Writing region close event to WAL at 1676325607948 (+5 ms)Running coprocessor post-close hooks at 1676325607960 (+12 ms)Closed at 1676325607960 2023-02-13 22:00:07,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:40899-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676325599411.f02c6dea934906492e781696bf263bb2. 2023-02-13 22:00:07,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableThree/f3962bf3439d5c7af4e285d7c3a11f11/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 22:00:07,964 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1837): Closed restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:07,964 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1557): Region close journal for f3962bf3439d5c7af4e285d7c3a11f11: Waiting for close lock at 1676325607957Running coprocessor pre-close hooks at 1676325607957Disabling compacts and flushes for region at 1676325607957Disabling writes for close at 1676325607958 (+1 ms)Writing region close event to WAL at 1676325607958Running coprocessor post-close hooks at 1676325607964 (+6 ms)Closed at 1676325607964 2023-02-13 22:00:07,964 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] handler.CloseRegionHandler(117): Closed restartTableThree,,1676325601271.f3962bf3439d5c7af4e285d7c3a11f11. 2023-02-13 22:00:07,966 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1603): Closing a9372b7742e56bdddc1c8be728e71577, disabling compactions & flushes 2023-02-13 22:00:07,966 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1625): Closing region restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:07,966 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1646): Waiting without time limit for close lock on restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:07,966 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1713): Acquired close lock on restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. after waiting 0 ms 2023-02-13 22:00:07,966 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1723): Updates disabled for region restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:07,971 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:37039-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.82 KB at sequenceid=38 (bloomFilter=false), to=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/.tmp/info/123ed49daa0f4c0ab85a7fd56766c57e 2023-02-13 22:00:07,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/default/restartTableTwo/a9372b7742e56bdddc1c8be728e71577/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 22:00:07,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1837): Closed restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:07,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1557): Region close journal for a9372b7742e56bdddc1c8be728e71577: Waiting for close lock at 1676325607964Running coprocessor pre-close hooks at 1676325607964Disabling compacts and flushes for region at 1676325607964Disabling writes for close at 1676325607966 (+2 ms)Writing region close event to WAL at 1676325607968 (+2 ms)Running coprocessor post-close hooks at 1676325607972 (+4 ms)Closed at 1676325607972 2023-02-13 22:00:07,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:37039-0] handler.CloseRegionHandler(117): Closed restartTableTwo,,1676325600656.a9372b7742e56bdddc1c8be728e71577. 2023-02-13 22:00:07,977 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/.tmp/info/123ed49daa0f4c0ab85a7fd56766c57e as hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/info/123ed49daa0f4c0ab85a7fd56766c57e 2023-02-13 22:00:07,984 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:37039-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/info/123ed49daa0f4c0ab85a7fd56766c57e, entries=32, sequenceid=38, filesize=8.5 K 2023-02-13 22:00:07,985 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(2947): Finished flush of dataSize ~3.82 KB/3908, heapSize ~6.40 KB/6552, currentSize=0 B/0 for 1588230740 in 41ms, sequenceid=38, compaction requested=false 2023-02-13 22:00:07,998 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:37039-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/data/hbase/meta/1588230740/recovered.edits/41.seqid, newMaxSeqId=41, maxSeqId=27 2023-02-13 22:00:07,998 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:37039-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 22:00:07,999 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 22:00:07,999 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:37039-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676325607942Running coprocessor pre-close hooks at 1676325607942Disabling compacts and flushes for region at 1676325607942Disabling writes for close at 1676325607944 (+2 ms)Obtaining lock to block concurrent updates at 1676325607944Preparing flush snapshotting stores in 1588230740 at 1676325607944Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=3908, getHeapSize=7032, getOffHeapSize=0, getCellsCount=32 at 1676325607944Flushing stores of hbase:meta,,1.1588230740 at 1676325607945 (+1 ms)Flushing 1588230740/info: creating writer at 1676325607945Flushing 1588230740/info: appending metadata at 1676325607954 (+9 ms)Flushing 1588230740/info: closing flushed file at 1676325607954Flushing 1588230740/info: reopening flushed file at 1676325607978 (+24 ms)Finished flush of dataSize ~3.82 KB/3908, heapSize ~6.40 KB/6552, currentSize=0 B/0 for 1588230740 in 41ms, sequenceid=38, compaction requested=false at 1676325607985 (+7 ms)Writing region close event to WAL at 1676325607994 (+9 ms)Running coprocessor post-close hooks at 1676325607998 (+4 ms)Closed at 1676325607999 (+1 ms) 2023-02-13 22:00:07,999 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:37039-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 22:00:08,034 INFO [regionserver/jenkins-hbase20:37039.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:00:08,034 INFO [regionserver/jenkins-hbase20:40899.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:00:08,132 INFO [regionserver/jenkins-hbase20:37039.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-13 22:00:08,132 INFO [regionserver/jenkins-hbase20:37039.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-13 22:00:08,135 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:45945-0x1015dda772d000d, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:08,135 INFO [RS:4;jenkins-hbase20:45945] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,45945,1676325605165; zookeeper connection closed. 2023-02-13 22:00:08,136 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@54e547fa rejected from java.util.concurrent.ThreadPoolExecutor@37041911[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:08,136 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@11ff39d7] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@11ff39d7 2023-02-13 22:00:08,136 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:45945-0x1015dda772d000d, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:08,136 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@25867ab rejected from java.util.concurrent.ThreadPoolExecutor@37041911[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:08,141 INFO [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,40899,1676325605280; all regions closed. 2023-02-13 22:00:08,143 INFO [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,37039,1676325604752; all regions closed. 2023-02-13 22:00:08,147 DEBUG [RS:5;jenkins-hbase20:40899] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs 2023-02-13 22:00:08,147 INFO [RS:5;jenkins-hbase20:40899] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C40899%2C1676325605280:(num 1676325605566) 2023-02-13 22:00:08,148 DEBUG [RS:5;jenkins-hbase20:40899] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:08,148 INFO [RS:5;jenkins-hbase20:40899] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:00:08,148 INFO [RS:5;jenkins-hbase20:40899] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:40899 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 22:00:08,148 INFO [RS:5;jenkins-hbase20:40899] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 22:00:08,148 INFO [regionserver/jenkins-hbase20:40899.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 22:00:08,148 INFO [RS:5;jenkins-hbase20:40899] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 22:00:08,148 INFO [RS:5;jenkins-hbase20:40899] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 22:00:08,149 INFO [RS:5;jenkins-hbase20:40899] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:40899 2023-02-13 22:00:08,150 DEBUG [RS:3;jenkins-hbase20:37039] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs 2023-02-13 22:00:08,150 INFO [RS:3;jenkins-hbase20:37039] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C37039%2C1676325604752.meta:.meta(num 1676325605545) 2023-02-13 22:00:08,151 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,40899,1676325605280 2023-02-13 22:00:08,151 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:08,151 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x1015dda772d000f, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,40899,1676325605280 2023-02-13 22:00:08,151 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@c61f922 rejected from java.util.concurrent.ThreadPoolExecutor@85e6c76[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:08,151 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,40899,1676325605280] 2023-02-13 22:00:08,151 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,40899,1676325605280; numProcessing=2 2023-02-13 22:00:08,153 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,40899,1676325605280 already deleted, retry=false 2023-02-13 22:00:08,153 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,40899,1676325605280 expired; onlineServers=1 2023-02-13 22:00:08,155 DEBUG [RS:3;jenkins-hbase20:37039] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/oldWALs 2023-02-13 22:00:08,155 INFO [RS:3;jenkins-hbase20:37039] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C37039%2C1676325604752:(num 1676325605258) 2023-02-13 22:00:08,155 DEBUG [RS:3;jenkins-hbase20:37039] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:08,155 INFO [RS:3;jenkins-hbase20:37039] regionserver.LeaseManager(133): Closed leases 2023-02-13 22:00:08,155 INFO [RS:3;jenkins-hbase20:37039] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:37039 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 22:00:08,155 INFO [regionserver/jenkins-hbase20:37039.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 22:00:08,156 INFO [RS:3;jenkins-hbase20:37039] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:37039 2023-02-13 22:00:08,157 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,37039,1676325604752 2023-02-13 22:00:08,157 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 22:00:08,158 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,37039,1676325604752] 2023-02-13 22:00:08,158 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,37039,1676325604752; numProcessing=3 2023-02-13 22:00:08,158 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,37039,1676325604752 already deleted, retry=false 2023-02-13 22:00:08,158 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,37039,1676325604752 expired; onlineServers=0 2023-02-13 22:00:08,159 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,46519,1676325604727' ***** 2023-02-13 22:00:08,159 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-13 22:00:08,159 DEBUG [M:1;jenkins-hbase20:46519] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5fa1d471, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-13 22:00:08,160 INFO [M:1;jenkins-hbase20:46519] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,46519,1676325604727 2023-02-13 22:00:08,160 INFO [M:1;jenkins-hbase20:46519] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,46519,1676325604727; all regions closed. 2023-02-13 22:00:08,160 DEBUG [M:1;jenkins-hbase20:46519] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 22:00:08,160 DEBUG [M:1;jenkins-hbase20:46519] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 22:00:08,160 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 22:00:08,160 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676325604956] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676325604956,5,FailOnTimeoutGroup] 2023-02-13 22:00:08,160 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676325604956] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676325604956,5,FailOnTimeoutGroup] 2023-02-13 22:00:08,160 DEBUG [M:1;jenkins-hbase20:46519] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 22:00:08,161 INFO [M:1;jenkins-hbase20:46519] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 22:00:08,161 INFO [M:1;jenkins-hbase20:46519] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 22:00:08,161 INFO [M:1;jenkins-hbase20:46519] hbase.ChoreService(369): Chore service for: master/jenkins-hbase20:0 had [] on shutdown 2023-02-13 22:00:08,161 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 22:00:08,161 DEBUG [M:1;jenkins-hbase20:46519] master.HMaster(1502): Stopping service threads 2023-02-13 22:00:08,161 INFO [M:1;jenkins-hbase20:46519] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 22:00:08,161 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 22:00:08,162 ERROR [M:1;jenkins-hbase20:46519] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 22:00:08,162 INFO [M:1;jenkins-hbase20:46519] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 22:00:08,162 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 22:00:08,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 22:00:08,162 DEBUG [M:1;jenkins-hbase20:46519] zookeeper.ZKUtil(398): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 22:00:08,162 WARN [M:1;jenkins-hbase20:46519] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 22:00:08,162 INFO [M:1;jenkins-hbase20:46519] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 22:00:08,163 INFO [M:1;jenkins-hbase20:46519] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 22:00:08,163 DEBUG [M:1;jenkins-hbase20:46519] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 22:00:08,163 INFO [M:1;jenkins-hbase20:46519] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:00:08,163 DEBUG [M:1;jenkins-hbase20:46519] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:00:08,163 DEBUG [M:1;jenkins-hbase20:46519] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 22:00:08,163 DEBUG [M:1;jenkins-hbase20:46519] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:00:08,163 INFO [M:1;jenkins-hbase20:46519] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.81 KB heapSize=48.05 KB 2023-02-13 22:00:08,176 INFO [M:1;jenkins-hbase20:46519] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.81 KB at sequenceid=246 (bloomFilter=true), to=hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/0b8fc85ff43a4f8eb69c2f3a65bb3ac8 2023-02-13 22:00:08,182 DEBUG [M:1;jenkins-hbase20:46519] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/0b8fc85ff43a4f8eb69c2f3a65bb3ac8 as hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/0b8fc85ff43a4f8eb69c2f3a65bb3ac8 2023-02-13 22:00:08,188 INFO [M:1;jenkins-hbase20:46519] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34137/user/jenkins/test-data/f3302b02-9a17-6679-7718-789c2b9455a2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/0b8fc85ff43a4f8eb69c2f3a65bb3ac8, entries=13, sequenceid=246, filesize=6.2 K 2023-02-13 22:00:08,189 INFO [M:1;jenkins-hbase20:46519] regionserver.HRegion(2947): Finished flush of dataSize ~38.81 KB/39743, heapSize ~48.04 KB/49192, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 26ms, sequenceid=246, compaction requested=false 2023-02-13 22:00:08,192 INFO [M:1;jenkins-hbase20:46519] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 22:00:08,192 DEBUG [M:1;jenkins-hbase20:46519] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676325608163Disabling compacts and flushes for region at 1676325608163Disabling writes for close at 1676325608163Obtaining lock to block concurrent updates at 1676325608163Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676325608163Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=39743, getHeapSize=49192, getOffHeapSize=0, getCellsCount=121 at 1676325608164 (+1 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676325608165 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676325608165Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676325608168 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676325608168Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676325608183 (+15 ms)Finished flush of dataSize ~38.81 KB/39743, heapSize ~48.04 KB/49192, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 26ms, sequenceid=246, compaction requested=false at 1676325608189 (+6 ms)Writing region close event to WAL at 1676325608192 (+3 ms)Closed at 1676325608192 2023-02-13 22:00:08,194 INFO [M:1;jenkins-hbase20:46519] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 22:00:08,194 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 22:00:08,195 INFO [M:1;jenkins-hbase20:46519] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:46519 2023-02-13 22:00:08,197 DEBUG [M:1;jenkins-hbase20:46519] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase20.apache.org,46519,1676325604727 already deleted, retry=false 2023-02-13 22:00:08,639 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:08,639 INFO [M:1;jenkins-hbase20:46519] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,46519,1676325604727; zookeeper connection closed. 2023-02-13 22:00:08,639 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3a15ed70 rejected from java.util.concurrent.ThreadPoolExecutor@3e2a7eea[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 20] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:08,640 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): master:46519-0x1015dda772d0009, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:08,640 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5ff24371 rejected from java.util.concurrent.ThreadPoolExecutor@3e2a7eea[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 20] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:08,739 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:08,739 INFO [RS:3;jenkins-hbase20:37039] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,37039,1676325604752; zookeeper connection closed. 2023-02-13 22:00:08,740 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@12696bc7 rejected from java.util.concurrent.ThreadPoolExecutor@1656fd52[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:08,740 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@34d15103] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@34d15103 2023-02-13 22:00:08,740 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:37039-0x1015dda772d000a, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:08,740 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3b23e232 rejected from java.util.concurrent.ThreadPoolExecutor@1656fd52[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:08,840 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x1015dda772d000f, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:08,840 INFO [RS:5;jenkins-hbase20:40899] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,40899,1676325605280; zookeeper connection closed. 2023-02-13 22:00:08,840 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7ff04383 rejected from java.util.concurrent.ThreadPoolExecutor@85e6c76[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:08,840 DEBUG [Listener at localhost.localdomain/39067-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x1015dda772d000f, quorum=127.0.0.1:53523, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 22:00:08,840 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5b47b299] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5b47b299 2023-02-13 22:00:08,840 ERROR [Listener at localhost.localdomain/39067-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@79df387 rejected from java.util.concurrent.ThreadPoolExecutor@85e6c76[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 22:00:08,840 INFO [Listener at localhost.localdomain/39067] util.JVMClusterUtil(335): Shutdown of 2 master(s) and 6 regionserver(s) complete 2023-02-13 22:00:08,841 WARN [Listener at localhost.localdomain/39067] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 22:00:08,846 INFO [Listener at localhost.localdomain/39067] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 22:00:08,950 WARN [BP-38254169-148.251.75.209-1676325597918 heartbeating to localhost.localdomain/127.0.0.1:34137] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 22:00:08,950 WARN [BP-38254169-148.251.75.209-1676325597918 heartbeating to localhost.localdomain/127.0.0.1:34137] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-38254169-148.251.75.209-1676325597918 (Datanode Uuid 0b4e47ea-7dd1-4885-807c-0a6cf357a49f) service to localhost.localdomain/127.0.0.1:34137 2023-02-13 22:00:08,951 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/cluster_42a35679-fe38-66b4-c507-4e77d8912d93/dfs/data/data5/current/BP-38254169-148.251.75.209-1676325597918] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 22:00:08,951 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/cluster_42a35679-fe38-66b4-c507-4e77d8912d93/dfs/data/data6/current/BP-38254169-148.251.75.209-1676325597918] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 22:00:08,953 WARN [Listener at localhost.localdomain/39067] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 22:00:08,960 INFO [Listener at localhost.localdomain/39067] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 22:00:09,065 WARN [BP-38254169-148.251.75.209-1676325597918 heartbeating to localhost.localdomain/127.0.0.1:34137] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 22:00:09,065 WARN [BP-38254169-148.251.75.209-1676325597918 heartbeating to localhost.localdomain/127.0.0.1:34137] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-38254169-148.251.75.209-1676325597918 (Datanode Uuid cbf28cde-5b32-4679-879d-8dad84d19144) service to localhost.localdomain/127.0.0.1:34137 2023-02-13 22:00:09,066 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/cluster_42a35679-fe38-66b4-c507-4e77d8912d93/dfs/data/data3/current/BP-38254169-148.251.75.209-1676325597918] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 22:00:09,066 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/cluster_42a35679-fe38-66b4-c507-4e77d8912d93/dfs/data/data4/current/BP-38254169-148.251.75.209-1676325597918] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 22:00:09,067 WARN [Listener at localhost.localdomain/39067] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 22:00:09,074 INFO [Listener at localhost.localdomain/39067] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 22:00:09,178 WARN [BP-38254169-148.251.75.209-1676325597918 heartbeating to localhost.localdomain/127.0.0.1:34137] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 22:00:09,178 WARN [BP-38254169-148.251.75.209-1676325597918 heartbeating to localhost.localdomain/127.0.0.1:34137] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-38254169-148.251.75.209-1676325597918 (Datanode Uuid 1d43f1a7-51ee-4916-a9b2-7f176ee12168) service to localhost.localdomain/127.0.0.1:34137 2023-02-13 22:00:09,178 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/cluster_42a35679-fe38-66b4-c507-4e77d8912d93/dfs/data/data1/current/BP-38254169-148.251.75.209-1676325597918] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 22:00:09,179 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a1e2b01-509e-985b-65ea-88c18ce7ac29/cluster_42a35679-fe38-66b4-c507-4e77d8912d93/dfs/data/data2/current/BP-38254169-148.251.75.209-1676325597918] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 22:00:09,190 INFO [Listener at localhost.localdomain/39067] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-02-13 22:00:09,304 INFO [Listener at localhost.localdomain/39067] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-13 22:00:09,334 INFO [Listener at localhost.localdomain/39067] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-02-13 22:00:09,347 INFO [Listener at localhost.localdomain/39067] hbase.ResourceChecker(175): after: master.TestRetainAssignmentOnRestartSplitWithoutZk#testRetainAssignmentOnClusterRestart Thread=156 (was 103) Potentially hanging thread: regionserver/jenkins-hbase20:40899.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:34137 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:34137 from jenkins.hfs.9 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.8@localhost.localdomain:34137 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:34137 from jenkins.hfs.10 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost.localdomain:34137 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RS-EventLoopGroup-14-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.9@localhost.localdomain:34137 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-7 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.11@localhost.localdomain:34137 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase20:45945.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: RS-EventLoopGroup-15-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-6 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:34137 from jenkins.hfs.7 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-15-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:34137 from jenkins.hfs.8 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-16-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost.localdomain:34137 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost.localdomain/39067 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.6@localhost.localdomain:34137 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:34137 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-8 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:34137 from jenkins.hfs.5 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:34137 from jenkins.hfs.6 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: LeaseRenewer:jenkins.hfs.10@localhost.localdomain:34137 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.7@localhost.localdomain:34137 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1368077305) connection to localhost.localdomain/127.0.0.1:34137 from jenkins.hfs.11 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-16-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase20:37039.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) - Thread LEAK? -, OpenFileDescriptor=584 (was 525) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=267 (was 261) - SystemLoadAverage LEAK? -, ProcessCount=164 (was 164), AvailableMemoryMB=6861 (was 7189)