2023-09-03 04:59:55,552 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082 2023-09-03 04:59:55,566 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.regionserver.wal.TestAsyncLogRolling timeout: 13 mins 2023-09-03 04:59:55,602 INFO [Time-limited test] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=189, ProcessCount=167, AvailableMemoryMB=9345 2023-09-03 04:59:55,608 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-09-03 04:59:55,609 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5, deleteOnExit=true 2023-09-03 04:59:55,609 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-09-03 04:59:55,609 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/test.cache.data in system properties and HBase conf 2023-09-03 04:59:55,610 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/hadoop.tmp.dir in system properties and HBase conf 2023-09-03 04:59:55,610 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/hadoop.log.dir in system properties and HBase conf 2023-09-03 04:59:55,611 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/mapreduce.cluster.local.dir in system properties and HBase conf 2023-09-03 04:59:55,611 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-09-03 04:59:55,611 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-09-03 04:59:55,718 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-09-03 04:59:56,113 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-09-03 04:59:56,117 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-09-03 04:59:56,117 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-09-03 04:59:56,118 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-09-03 04:59:56,118 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-03 04:59:56,118 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-09-03 04:59:56,119 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-09-03 04:59:56,119 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-03 04:59:56,119 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-03 04:59:56,119 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-09-03 04:59:56,120 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/nfs.dump.dir in system properties and HBase conf 2023-09-03 04:59:56,120 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/java.io.tmpdir in system properties and HBase conf 2023-09-03 04:59:56,120 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-03 04:59:56,120 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-09-03 04:59:56,121 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-09-03 04:59:56,595 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-03 04:59:56,599 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-03 04:59:56,873 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-09-03 04:59:57,028 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-09-03 04:59:57,042 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 04:59:57,076 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 04:59:57,107 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/java.io.tmpdir/Jetty_localhost_44323_hdfs____9tolpa/webapp 2023-09-03 04:59:57,232 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44323 2023-09-03 04:59:57,251 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-03 04:59:57,252 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-03 04:59:57,749 WARN [Listener at localhost/39237] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 04:59:57,819 WARN [Listener at localhost/39237] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-03 04:59:57,837 WARN [Listener at localhost/39237] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 04:59:57,843 INFO [Listener at localhost/39237] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 04:59:57,847 INFO [Listener at localhost/39237] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/java.io.tmpdir/Jetty_localhost_35547_datanode____5sijzx/webapp 2023-09-03 04:59:57,942 INFO [Listener at localhost/39237] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35547 2023-09-03 04:59:58,246 WARN [Listener at localhost/39647] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 04:59:58,255 WARN [Listener at localhost/39647] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-03 04:59:58,258 WARN [Listener at localhost/39647] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 04:59:58,261 INFO [Listener at localhost/39647] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 04:59:58,267 INFO [Listener at localhost/39647] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/java.io.tmpdir/Jetty_localhost_44063_datanode____.ebq97w/webapp 2023-09-03 04:59:58,362 INFO [Listener at localhost/39647] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44063 2023-09-03 04:59:58,371 WARN [Listener at localhost/39895] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 04:59:58,681 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x914e02f8e6402541: Processing first storage report for DS-7d16f019-67b5-49b5-b8d9-77cd2d44437e from datanode 159899f9-8186-4c2d-ad8e-cecc5127279c 2023-09-03 04:59:58,683 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x914e02f8e6402541: from storage DS-7d16f019-67b5-49b5-b8d9-77cd2d44437e node DatanodeRegistration(127.0.0.1:40809, datanodeUuid=159899f9-8186-4c2d-ad8e-cecc5127279c, infoPort=43333, infoSecurePort=0, ipcPort=39647, storageInfo=lv=-57;cid=testClusterID;nsid=1833191466;c=1693717196670), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-09-03 04:59:58,683 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x42edf7d750469c3b: Processing first storage report for DS-ea21ea74-aa1b-49ab-954b-fbb16cc8a608 from datanode 36e45ea4-2bce-487a-965e-197048fe474c 2023-09-03 04:59:58,683 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x42edf7d750469c3b: from storage DS-ea21ea74-aa1b-49ab-954b-fbb16cc8a608 node DatanodeRegistration(127.0.0.1:45159, datanodeUuid=36e45ea4-2bce-487a-965e-197048fe474c, infoPort=39495, infoSecurePort=0, ipcPort=39895, storageInfo=lv=-57;cid=testClusterID;nsid=1833191466;c=1693717196670), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 04:59:58,683 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x914e02f8e6402541: Processing first storage report for DS-0a97c607-b024-4448-920a-13176a55ff03 from datanode 159899f9-8186-4c2d-ad8e-cecc5127279c 2023-09-03 04:59:58,684 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x914e02f8e6402541: from storage DS-0a97c607-b024-4448-920a-13176a55ff03 node DatanodeRegistration(127.0.0.1:40809, datanodeUuid=159899f9-8186-4c2d-ad8e-cecc5127279c, infoPort=43333, infoSecurePort=0, ipcPort=39647, storageInfo=lv=-57;cid=testClusterID;nsid=1833191466;c=1693717196670), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 04:59:58,684 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x42edf7d750469c3b: Processing first storage report for DS-2ada62b9-9737-4c8e-aa2c-77d396ec62a2 from datanode 36e45ea4-2bce-487a-965e-197048fe474c 2023-09-03 04:59:58,684 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x42edf7d750469c3b: from storage DS-2ada62b9-9737-4c8e-aa2c-77d396ec62a2 node DatanodeRegistration(127.0.0.1:45159, datanodeUuid=36e45ea4-2bce-487a-965e-197048fe474c, infoPort=39495, infoSecurePort=0, ipcPort=39895, storageInfo=lv=-57;cid=testClusterID;nsid=1833191466;c=1693717196670), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 04:59:58,755 DEBUG [Listener at localhost/39895] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082 2023-09-03 04:59:58,817 INFO [Listener at localhost/39895] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5/zookeeper_0, clientPort=51414, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-09-03 04:59:58,831 INFO [Listener at localhost/39895] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=51414 2023-09-03 04:59:58,842 INFO [Listener at localhost/39895] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 04:59:58,845 INFO [Listener at localhost/39895] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 04:59:59,513 INFO [Listener at localhost/39895] util.FSUtils(471): Created version file at hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610 with version=8 2023-09-03 04:59:59,513 INFO [Listener at localhost/39895] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/hbase-staging 2023-09-03 04:59:59,828 INFO [Listener at localhost/39895] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-09-03 05:00:00,307 INFO [Listener at localhost/39895] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-09-03 05:00:00,339 INFO [Listener at localhost/39895] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:00:00,340 INFO [Listener at localhost/39895] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-03 05:00:00,340 INFO [Listener at localhost/39895] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-03 05:00:00,340 INFO [Listener at localhost/39895] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:00:00,340 INFO [Listener at localhost/39895] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-03 05:00:00,483 INFO [Listener at localhost/39895] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-09-03 05:00:00,558 DEBUG [Listener at localhost/39895] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-09-03 05:00:00,651 INFO [Listener at localhost/39895] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43827 2023-09-03 05:00:00,661 INFO [Listener at localhost/39895] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:00:00,663 INFO [Listener at localhost/39895] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:00:00,684 INFO [Listener at localhost/39895] zookeeper.RecoverableZooKeeper(93): Process identifier=master:43827 connecting to ZooKeeper ensemble=127.0.0.1:51414 2023-09-03 05:00:00,721 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:438270x0, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-03 05:00:00,724 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:43827-0x1007028af780000 connected 2023-09-03 05:00:00,745 DEBUG [Listener at localhost/39895] zookeeper.ZKUtil(164): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-03 05:00:00,746 DEBUG [Listener at localhost/39895] zookeeper.ZKUtil(164): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:00:00,749 DEBUG [Listener at localhost/39895] zookeeper.ZKUtil(164): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-03 05:00:00,757 DEBUG [Listener at localhost/39895] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43827 2023-09-03 05:00:00,757 DEBUG [Listener at localhost/39895] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43827 2023-09-03 05:00:00,757 DEBUG [Listener at localhost/39895] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43827 2023-09-03 05:00:00,758 DEBUG [Listener at localhost/39895] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43827 2023-09-03 05:00:00,758 DEBUG [Listener at localhost/39895] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43827 2023-09-03 05:00:00,764 INFO [Listener at localhost/39895] master.HMaster(444): hbase.rootdir=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610, hbase.cluster.distributed=false 2023-09-03 05:00:00,832 INFO [Listener at localhost/39895] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-09-03 05:00:00,833 INFO [Listener at localhost/39895] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:00:00,833 INFO [Listener at localhost/39895] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-03 05:00:00,833 INFO [Listener at localhost/39895] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-03 05:00:00,833 INFO [Listener at localhost/39895] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:00:00,833 INFO [Listener at localhost/39895] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-03 05:00:00,838 INFO [Listener at localhost/39895] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-09-03 05:00:00,841 INFO [Listener at localhost/39895] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38963 2023-09-03 05:00:00,843 INFO [Listener at localhost/39895] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-03 05:00:00,849 DEBUG [Listener at localhost/39895] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-09-03 05:00:00,850 INFO [Listener at localhost/39895] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:00:00,852 INFO [Listener at localhost/39895] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:00:00,854 INFO [Listener at localhost/39895] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38963 connecting to ZooKeeper ensemble=127.0.0.1:51414 2023-09-03 05:00:00,857 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): regionserver:389630x0, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-03 05:00:00,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:38963-0x1007028af780001 connected 2023-09-03 05:00:00,858 DEBUG [Listener at localhost/39895] zookeeper.ZKUtil(164): regionserver:38963-0x1007028af780001, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-03 05:00:00,860 DEBUG [Listener at localhost/39895] zookeeper.ZKUtil(164): regionserver:38963-0x1007028af780001, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:00:00,861 DEBUG [Listener at localhost/39895] zookeeper.ZKUtil(164): regionserver:38963-0x1007028af780001, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-03 05:00:00,861 DEBUG [Listener at localhost/39895] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38963 2023-09-03 05:00:00,861 DEBUG [Listener at localhost/39895] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38963 2023-09-03 05:00:00,862 DEBUG [Listener at localhost/39895] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38963 2023-09-03 05:00:00,862 DEBUG [Listener at localhost/39895] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38963 2023-09-03 05:00:00,863 DEBUG [Listener at localhost/39895] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38963 2023-09-03 05:00:00,864 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,43827,1693717199663 2023-09-03 05:00:00,873 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-03 05:00:00,874 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,43827,1693717199663 2023-09-03 05:00:00,894 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-03 05:00:00,894 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): regionserver:38963-0x1007028af780001, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-03 05:00:00,894 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:00,895 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-03 05:00:00,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,43827,1693717199663 from backup master directory 2023-09-03 05:00:00,896 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-03 05:00:00,899 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,43827,1693717199663 2023-09-03 05:00:00,899 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-03 05:00:00,900 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-03 05:00:00,900 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,43827,1693717199663 2023-09-03 05:00:00,902 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-09-03 05:00:00,904 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-09-03 05:00:00,986 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/hbase.id with ID: 07e75569-90b3-4595-94d1-1c0c26761db2 2023-09-03 05:00:01,029 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:00:01,043 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:01,084 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x73eb9cf3 to 127.0.0.1:51414 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-03 05:00:01,119 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@ef5bede, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-03 05:00:01,144 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-03 05:00:01,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-09-03 05:00:01,155 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-03 05:00:01,191 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/data/master/store-tmp 2023-09-03 05:00:01,222 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:01,222 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-03 05:00:01,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:00:01,223 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:00:01,223 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-03 05:00:01,223 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:00:01,223 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:00:01,223 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-03 05:00:01,224 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/WALs/jenkins-hbase4.apache.org,43827,1693717199663 2023-09-03 05:00:01,246 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43827%2C1693717199663, suffix=, logDir=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/WALs/jenkins-hbase4.apache.org,43827,1693717199663, archiveDir=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/oldWALs, maxLogs=10 2023-09-03 05:00:01,281 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-09-03 05:00:01,281 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-09-03 05:00:01,283 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:190) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:719) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:01,315 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:417) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:01,317 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40809,DS-7d16f019-67b5-49b5-b8d9-77cd2d44437e,DISK] 2023-09-03 05:00:01,317 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45159,DS-ea21ea74-aa1b-49ab-954b-fbb16cc8a608,DISK] 2023-09-03 05:00:01,324 DEBUG [RS-EventLoopGroup-3-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:01,388 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/WALs/jenkins-hbase4.apache.org,43827,1693717199663/jenkins-hbase4.apache.org%2C43827%2C1693717199663.1693717201258 2023-09-03 05:00:01,388 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45159,DS-ea21ea74-aa1b-49ab-954b-fbb16cc8a608,DISK], DatanodeInfoWithStorage[127.0.0.1:40809,DS-7d16f019-67b5-49b5-b8d9-77cd2d44437e,DISK]] 2023-09-03 05:00:01,389 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-09-03 05:00:01,389 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:01,392 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:00:01,394 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:00:01,450 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:00:01,458 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-09-03 05:00:01,482 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-09-03 05:00:01,495 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:01,500 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:00:01,502 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:00:01,518 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:00:01,522 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-03 05:00:01,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=820323, jitterRate=0.04309585690498352}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-03 05:00:01,523 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-03 05:00:01,524 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-09-03 05:00:01,543 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-09-03 05:00:01,543 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-09-03 05:00:01,545 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-09-03 05:00:01,547 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 1 msec 2023-09-03 05:00:01,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 32 msec 2023-09-03 05:00:01,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-09-03 05:00:01,605 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-09-03 05:00:01,611 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-09-03 05:00:01,643 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-09-03 05:00:01,648 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-09-03 05:00:01,651 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-09-03 05:00:01,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-09-03 05:00:01,664 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-09-03 05:00:01,667 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:01,669 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-09-03 05:00:01,669 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-09-03 05:00:01,681 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-09-03 05:00:01,687 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): regionserver:38963-0x1007028af780001, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-03 05:00:01,687 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-03 05:00:01,687 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:01,688 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,43827,1693717199663, sessionid=0x1007028af780000, setting cluster-up flag (Was=false) 2023-09-03 05:00:01,703 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:01,709 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-09-03 05:00:01,710 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43827,1693717199663 2023-09-03 05:00:01,716 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:01,722 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-09-03 05:00:01,724 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43827,1693717199663 2023-09-03 05:00:01,726 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/.hbase-snapshot/.tmp 2023-09-03 05:00:01,767 INFO [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(951): ClusterId : 07e75569-90b3-4595-94d1-1c0c26761db2 2023-09-03 05:00:01,771 DEBUG [RS:0;jenkins-hbase4:38963] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-09-03 05:00:01,776 DEBUG [RS:0;jenkins-hbase4:38963] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-09-03 05:00:01,776 DEBUG [RS:0;jenkins-hbase4:38963] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-09-03 05:00:01,780 DEBUG [RS:0;jenkins-hbase4:38963] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-09-03 05:00:01,781 DEBUG [RS:0;jenkins-hbase4:38963] zookeeper.ReadOnlyZKClient(139): Connect 0x669fb05d to 127.0.0.1:51414 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-03 05:00:01,786 DEBUG [RS:0;jenkins-hbase4:38963] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@685842fe, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-03 05:00:01,786 DEBUG [RS:0;jenkins-hbase4:38963] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c67db2d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-03 05:00:01,816 DEBUG [RS:0;jenkins-hbase4:38963] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:38963 2023-09-03 05:00:01,820 INFO [RS:0;jenkins-hbase4:38963] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-09-03 05:00:01,820 INFO [RS:0;jenkins-hbase4:38963] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-09-03 05:00:01,820 DEBUG [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(1022): About to register with Master. 2023-09-03 05:00:01,823 INFO [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,43827,1693717199663 with isa=jenkins-hbase4.apache.org/172.31.14.131:38963, startcode=1693717200832 2023-09-03 05:00:01,843 DEBUG [RS:0;jenkins-hbase4:38963] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-09-03 05:00:01,862 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-09-03 05:00:01,873 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:00:01,873 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:00:01,873 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:00:01,873 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:00:01,873 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-09-03 05:00:01,873 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:01,874 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-03 05:00:01,874 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:01,877 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693717231877 2023-09-03 05:00:01,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-09-03 05:00:01,887 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-09-03 05:00:01,887 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-09-03 05:00:01,894 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-09-03 05:00:01,895 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-03 05:00:01,901 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49605, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-09-03 05:00:01,904 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-09-03 05:00:01,905 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-09-03 05:00:01,905 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-09-03 05:00:01,906 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-09-03 05:00:01,907 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:01,911 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=43827] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-03 05:00:01,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-09-03 05:00:01,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-09-03 05:00:01,915 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-09-03 05:00:01,918 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-09-03 05:00:01,919 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-09-03 05:00:01,923 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693717201923,5,FailOnTimeoutGroup] 2023-09-03 05:00:01,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693717201924,5,FailOnTimeoutGroup] 2023-09-03 05:00:01,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:01,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-09-03 05:00:01,929 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:01,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:01,936 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-03 05:00:01,937 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-03 05:00:01,937 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610 2023-09-03 05:00:01,954 DEBUG [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(2830): Master is not running yet 2023-09-03 05:00:01,954 WARN [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-09-03 05:00:01,956 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:01,959 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-03 05:00:01,962 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/info 2023-09-03 05:00:01,962 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-03 05:00:01,963 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:01,964 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-03 05:00:01,966 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/rep_barrier 2023-09-03 05:00:01,967 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-03 05:00:01,968 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:01,968 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-03 05:00:01,971 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/table 2023-09-03 05:00:01,972 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-03 05:00:01,973 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:01,974 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740 2023-09-03 05:00:01,975 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740 2023-09-03 05:00:01,979 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-03 05:00:01,982 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-03 05:00:01,986 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-03 05:00:01,987 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=764699, jitterRate=-0.027636051177978516}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-03 05:00:01,987 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-03 05:00:01,987 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-03 05:00:01,987 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-03 05:00:01,987 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-03 05:00:01,987 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-03 05:00:01,987 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-03 05:00:01,988 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-03 05:00:01,988 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-03 05:00:01,993 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-09-03 05:00:01,993 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-09-03 05:00:02,003 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-09-03 05:00:02,015 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-09-03 05:00:02,018 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-09-03 05:00:02,056 INFO [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,43827,1693717199663 with isa=jenkins-hbase4.apache.org/172.31.14.131:38963, startcode=1693717200832 2023-09-03 05:00:02,063 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=43827] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38963,1693717200832 2023-09-03 05:00:02,070 DEBUG [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610 2023-09-03 05:00:02,070 DEBUG [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:39237 2023-09-03 05:00:02,070 DEBUG [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-09-03 05:00:02,076 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-03 05:00:02,078 DEBUG [RS:0;jenkins-hbase4:38963] zookeeper.ZKUtil(162): regionserver:38963-0x1007028af780001, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38963,1693717200832 2023-09-03 05:00:02,078 WARN [RS:0;jenkins-hbase4:38963] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-03 05:00:02,079 INFO [RS:0;jenkins-hbase4:38963] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-03 05:00:02,079 DEBUG [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(1948): logDir=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832 2023-09-03 05:00:02,081 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38963,1693717200832] 2023-09-03 05:00:02,090 DEBUG [RS:0;jenkins-hbase4:38963] zookeeper.ZKUtil(162): regionserver:38963-0x1007028af780001, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38963,1693717200832 2023-09-03 05:00:02,107 DEBUG [RS:0;jenkins-hbase4:38963] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-09-03 05:00:02,117 INFO [RS:0;jenkins-hbase4:38963] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-09-03 05:00:02,142 INFO [RS:0;jenkins-hbase4:38963] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-09-03 05:00:02,145 INFO [RS:0;jenkins-hbase4:38963] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-09-03 05:00:02,146 INFO [RS:0;jenkins-hbase4:38963] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:02,147 INFO [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-09-03 05:00:02,154 INFO [RS:0;jenkins-hbase4:38963] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:02,154 DEBUG [RS:0;jenkins-hbase4:38963] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:02,155 DEBUG [RS:0;jenkins-hbase4:38963] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:02,155 DEBUG [RS:0;jenkins-hbase4:38963] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:02,155 DEBUG [RS:0;jenkins-hbase4:38963] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:02,155 DEBUG [RS:0;jenkins-hbase4:38963] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:02,155 DEBUG [RS:0;jenkins-hbase4:38963] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-03 05:00:02,155 DEBUG [RS:0;jenkins-hbase4:38963] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:02,156 DEBUG [RS:0;jenkins-hbase4:38963] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:02,156 DEBUG [RS:0;jenkins-hbase4:38963] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:02,156 DEBUG [RS:0;jenkins-hbase4:38963] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:02,157 INFO [RS:0;jenkins-hbase4:38963] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:02,157 INFO [RS:0;jenkins-hbase4:38963] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:02,157 INFO [RS:0;jenkins-hbase4:38963] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:02,170 DEBUG [jenkins-hbase4:43827] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-09-03 05:00:02,178 INFO [RS:0;jenkins-hbase4:38963] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-09-03 05:00:02,179 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38963,1693717200832, state=OPENING 2023-09-03 05:00:02,181 INFO [RS:0;jenkins-hbase4:38963] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38963,1693717200832-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:02,189 DEBUG [PEWorker-2] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-09-03 05:00:02,192 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:02,192 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-03 05:00:02,197 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38963,1693717200832}] 2023-09-03 05:00:02,202 INFO [RS:0;jenkins-hbase4:38963] regionserver.Replication(203): jenkins-hbase4.apache.org,38963,1693717200832 started 2023-09-03 05:00:02,202 INFO [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,38963,1693717200832, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38963, sessionid=0x1007028af780001 2023-09-03 05:00:02,202 DEBUG [RS:0;jenkins-hbase4:38963] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-09-03 05:00:02,202 DEBUG [RS:0;jenkins-hbase4:38963] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38963,1693717200832 2023-09-03 05:00:02,202 DEBUG [RS:0;jenkins-hbase4:38963] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38963,1693717200832' 2023-09-03 05:00:02,202 DEBUG [RS:0;jenkins-hbase4:38963] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-03 05:00:02,203 DEBUG [RS:0;jenkins-hbase4:38963] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-03 05:00:02,203 DEBUG [RS:0;jenkins-hbase4:38963] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-09-03 05:00:02,203 DEBUG [RS:0;jenkins-hbase4:38963] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-09-03 05:00:02,203 DEBUG [RS:0;jenkins-hbase4:38963] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38963,1693717200832 2023-09-03 05:00:02,203 DEBUG [RS:0;jenkins-hbase4:38963] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38963,1693717200832' 2023-09-03 05:00:02,204 DEBUG [RS:0;jenkins-hbase4:38963] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-09-03 05:00:02,204 DEBUG [RS:0;jenkins-hbase4:38963] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-09-03 05:00:02,205 DEBUG [RS:0;jenkins-hbase4:38963] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-09-03 05:00:02,205 INFO [RS:0;jenkins-hbase4:38963] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-09-03 05:00:02,205 INFO [RS:0;jenkins-hbase4:38963] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-09-03 05:00:02,317 INFO [RS:0;jenkins-hbase4:38963] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38963%2C1693717200832, suffix=, logDir=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832, archiveDir=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/oldWALs, maxLogs=32 2023-09-03 05:00:02,337 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40809,DS-7d16f019-67b5-49b5-b8d9-77cd2d44437e,DISK] 2023-09-03 05:00:02,337 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45159,DS-ea21ea74-aa1b-49ab-954b-fbb16cc8a608,DISK] 2023-09-03 05:00:02,341 INFO [RS:0;jenkins-hbase4:38963] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.1693717202319 2023-09-03 05:00:02,341 DEBUG [RS:0;jenkins-hbase4:38963] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40809,DS-7d16f019-67b5-49b5-b8d9-77cd2d44437e,DISK], DatanodeInfoWithStorage[127.0.0.1:45159,DS-ea21ea74-aa1b-49ab-954b-fbb16cc8a608,DISK]] 2023-09-03 05:00:02,382 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,38963,1693717200832 2023-09-03 05:00:02,384 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-09-03 05:00:02,388 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55418, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-09-03 05:00:02,400 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-09-03 05:00:02,401 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-03 05:00:02,405 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38963%2C1693717200832.meta, suffix=.meta, logDir=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832, archiveDir=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/oldWALs, maxLogs=32 2023-09-03 05:00:02,430 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45159,DS-ea21ea74-aa1b-49ab-954b-fbb16cc8a608,DISK] 2023-09-03 05:00:02,430 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40809,DS-7d16f019-67b5-49b5-b8d9-77cd2d44437e,DISK] 2023-09-03 05:00:02,434 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.meta.1693717202407.meta 2023-09-03 05:00:02,435 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45159,DS-ea21ea74-aa1b-49ab-954b-fbb16cc8a608,DISK], DatanodeInfoWithStorage[127.0.0.1:40809,DS-7d16f019-67b5-49b5-b8d9-77cd2d44437e,DISK]] 2023-09-03 05:00:02,435 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-09-03 05:00:02,437 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-09-03 05:00:02,454 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-09-03 05:00:02,460 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-09-03 05:00:02,467 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-09-03 05:00:02,467 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:02,467 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-09-03 05:00:02,467 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-09-03 05:00:02,470 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-03 05:00:02,472 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/info 2023-09-03 05:00:02,472 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/info 2023-09-03 05:00:02,473 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-03 05:00:02,474 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:02,474 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-03 05:00:02,475 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/rep_barrier 2023-09-03 05:00:02,475 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/rep_barrier 2023-09-03 05:00:02,476 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-03 05:00:02,477 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:02,477 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-03 05:00:02,478 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/table 2023-09-03 05:00:02,478 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/table 2023-09-03 05:00:02,479 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-03 05:00:02,480 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:02,481 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740 2023-09-03 05:00:02,484 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740 2023-09-03 05:00:02,487 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-03 05:00:02,489 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-03 05:00:02,491 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=737135, jitterRate=-0.06268547475337982}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-03 05:00:02,491 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-03 05:00:02,501 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693717202374 2023-09-03 05:00:02,519 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-09-03 05:00:02,520 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-09-03 05:00:02,520 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38963,1693717200832, state=OPEN 2023-09-03 05:00:02,523 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-09-03 05:00:02,523 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-03 05:00:02,530 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-09-03 05:00:02,530 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38963,1693717200832 in 326 msec 2023-09-03 05:00:02,535 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-09-03 05:00:02,535 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 528 msec 2023-09-03 05:00:02,540 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 748 msec 2023-09-03 05:00:02,540 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693717202540, completionTime=-1 2023-09-03 05:00:02,540 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-09-03 05:00:02,541 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-09-03 05:00:02,603 DEBUG [hconnection-0x26da0011-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-03 05:00:02,607 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55420, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-03 05:00:02,631 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-09-03 05:00:02,631 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693717262631 2023-09-03 05:00:02,631 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693717322631 2023-09-03 05:00:02,631 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 90 msec 2023-09-03 05:00:02,663 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43827,1693717199663-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:02,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43827,1693717199663-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:02,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43827,1693717199663-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:02,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:43827, period=300000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:02,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:02,673 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-09-03 05:00:02,681 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-09-03 05:00:02,682 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-03 05:00:02,693 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-09-03 05:00:02,696 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-09-03 05:00:02,698 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-03 05:00:02,721 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/.tmp/data/hbase/namespace/64ccc3a1ad492104480fb5b883ad071e 2023-09-03 05:00:02,724 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/.tmp/data/hbase/namespace/64ccc3a1ad492104480fb5b883ad071e empty. 2023-09-03 05:00:02,724 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/.tmp/data/hbase/namespace/64ccc3a1ad492104480fb5b883ad071e 2023-09-03 05:00:02,725 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-09-03 05:00:02,755 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-09-03 05:00:02,758 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 64ccc3a1ad492104480fb5b883ad071e, NAME => 'hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/.tmp 2023-09-03 05:00:02,777 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:02,777 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 64ccc3a1ad492104480fb5b883ad071e, disabling compactions & flushes 2023-09-03 05:00:02,778 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e. 2023-09-03 05:00:02,778 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e. 2023-09-03 05:00:02,778 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e. after waiting 0 ms 2023-09-03 05:00:02,778 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e. 2023-09-03 05:00:02,778 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e. 2023-09-03 05:00:02,778 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 64ccc3a1ad492104480fb5b883ad071e: 2023-09-03 05:00:02,783 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-09-03 05:00:02,800 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693717202786"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693717202786"}]},"ts":"1693717202786"} 2023-09-03 05:00:02,831 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-03 05:00:02,834 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-03 05:00:02,840 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693717202834"}]},"ts":"1693717202834"} 2023-09-03 05:00:02,844 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-09-03 05:00:02,858 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=64ccc3a1ad492104480fb5b883ad071e, ASSIGN}] 2023-09-03 05:00:02,861 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=64ccc3a1ad492104480fb5b883ad071e, ASSIGN 2023-09-03 05:00:02,863 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=64ccc3a1ad492104480fb5b883ad071e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38963,1693717200832; forceNewPlan=false, retain=false 2023-09-03 05:00:03,015 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=64ccc3a1ad492104480fb5b883ad071e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38963,1693717200832 2023-09-03 05:00:03,015 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693717203014"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693717203014"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693717203014"}]},"ts":"1693717203014"} 2023-09-03 05:00:03,020 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 64ccc3a1ad492104480fb5b883ad071e, server=jenkins-hbase4.apache.org,38963,1693717200832}] 2023-09-03 05:00:03,183 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e. 2023-09-03 05:00:03,184 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 64ccc3a1ad492104480fb5b883ad071e, NAME => 'hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e.', STARTKEY => '', ENDKEY => ''} 2023-09-03 05:00:03,186 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 64ccc3a1ad492104480fb5b883ad071e 2023-09-03 05:00:03,186 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:03,186 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 64ccc3a1ad492104480fb5b883ad071e 2023-09-03 05:00:03,186 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 64ccc3a1ad492104480fb5b883ad071e 2023-09-03 05:00:03,188 INFO [StoreOpener-64ccc3a1ad492104480fb5b883ad071e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 64ccc3a1ad492104480fb5b883ad071e 2023-09-03 05:00:03,191 DEBUG [StoreOpener-64ccc3a1ad492104480fb5b883ad071e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/namespace/64ccc3a1ad492104480fb5b883ad071e/info 2023-09-03 05:00:03,191 DEBUG [StoreOpener-64ccc3a1ad492104480fb5b883ad071e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/namespace/64ccc3a1ad492104480fb5b883ad071e/info 2023-09-03 05:00:03,191 INFO [StoreOpener-64ccc3a1ad492104480fb5b883ad071e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 64ccc3a1ad492104480fb5b883ad071e columnFamilyName info 2023-09-03 05:00:03,192 INFO [StoreOpener-64ccc3a1ad492104480fb5b883ad071e-1] regionserver.HStore(310): Store=64ccc3a1ad492104480fb5b883ad071e/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:03,194 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/namespace/64ccc3a1ad492104480fb5b883ad071e 2023-09-03 05:00:03,195 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/namespace/64ccc3a1ad492104480fb5b883ad071e 2023-09-03 05:00:03,199 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 64ccc3a1ad492104480fb5b883ad071e 2023-09-03 05:00:03,205 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/namespace/64ccc3a1ad492104480fb5b883ad071e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-03 05:00:03,206 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 64ccc3a1ad492104480fb5b883ad071e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=716985, jitterRate=-0.08830668032169342}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-03 05:00:03,206 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 64ccc3a1ad492104480fb5b883ad071e: 2023-09-03 05:00:03,208 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e., pid=6, masterSystemTime=1693717203174 2023-09-03 05:00:03,213 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e. 2023-09-03 05:00:03,213 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e. 2023-09-03 05:00:03,216 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=64ccc3a1ad492104480fb5b883ad071e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38963,1693717200832 2023-09-03 05:00:03,216 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693717203215"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693717203215"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693717203215"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693717203215"}]},"ts":"1693717203215"} 2023-09-03 05:00:03,228 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-09-03 05:00:03,228 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 64ccc3a1ad492104480fb5b883ad071e, server=jenkins-hbase4.apache.org,38963,1693717200832 in 203 msec 2023-09-03 05:00:03,233 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-09-03 05:00:03,233 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=64ccc3a1ad492104480fb5b883ad071e, ASSIGN in 370 msec 2023-09-03 05:00:03,235 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-03 05:00:03,236 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693717203236"}]},"ts":"1693717203236"} 2023-09-03 05:00:03,240 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-09-03 05:00:03,246 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-09-03 05:00:03,249 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 562 msec 2023-09-03 05:00:03,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-09-03 05:00:03,299 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-09-03 05:00:03,300 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:03,343 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-09-03 05:00:03,363 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-03 05:00:03,370 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 37 msec 2023-09-03 05:00:03,377 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-09-03 05:00:03,391 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-03 05:00:03,397 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 19 msec 2023-09-03 05:00:03,403 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-09-03 05:00:03,407 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-09-03 05:00:03,407 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.507sec 2023-09-03 05:00:03,410 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-09-03 05:00:03,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-09-03 05:00:03,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-09-03 05:00:03,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43827,1693717199663-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-09-03 05:00:03,413 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43827,1693717199663-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-09-03 05:00:03,423 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-09-03 05:00:03,472 DEBUG [Listener at localhost/39895] zookeeper.ReadOnlyZKClient(139): Connect 0x78deaf86 to 127.0.0.1:51414 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-03 05:00:03,477 DEBUG [Listener at localhost/39895] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@42bfd752, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-03 05:00:03,535 DEBUG [hconnection-0xfaaaf20-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-03 05:00:03,547 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55434, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-03 05:00:03,560 INFO [Listener at localhost/39895] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,43827,1693717199663 2023-09-03 05:00:03,561 INFO [Listener at localhost/39895] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:00:03,570 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-09-03 05:00:03,570 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:03,571 INFO [Listener at localhost/39895] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-09-03 05:00:03,585 WARN [Listener at localhost/39895] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-03 05:00:03,587 WARN [Listener at localhost/39895] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 05:00:03,588 INFO [Listener at localhost/39895] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 05:00:03,593 INFO [Listener at localhost/39895] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/java.io.tmpdir/Jetty_localhost_42675_datanode____.7z237n/webapp 2023-09-03 05:00:03,693 INFO [Listener at localhost/39895] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42675 2023-09-03 05:00:03,702 WARN [Listener at localhost/39133] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 05:00:03,730 WARN [Listener at localhost/39133] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-03 05:00:03,733 WARN [Listener at localhost/39133] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 05:00:03,735 INFO [Listener at localhost/39133] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 05:00:03,740 INFO [Listener at localhost/39133] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/java.io.tmpdir/Jetty_localhost_33813_datanode____.3e3vj/webapp 2023-09-03 05:00:03,819 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa398c91ac93c0823: Processing first storage report for DS-a31c2e55-d189-405f-8eed-6648ac367e8a from datanode c5b4826a-c83e-4331-8813-9b9896231180 2023-09-03 05:00:03,819 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa398c91ac93c0823: from storage DS-a31c2e55-d189-405f-8eed-6648ac367e8a node DatanodeRegistration(127.0.0.1:37011, datanodeUuid=c5b4826a-c83e-4331-8813-9b9896231180, infoPort=32797, infoSecurePort=0, ipcPort=39133, storageInfo=lv=-57;cid=testClusterID;nsid=1833191466;c=1693717196670), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:00:03,819 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa398c91ac93c0823: Processing first storage report for DS-3e74acb0-46f7-45bd-9d2a-13fca7c30ef7 from datanode c5b4826a-c83e-4331-8813-9b9896231180 2023-09-03 05:00:03,819 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa398c91ac93c0823: from storage DS-3e74acb0-46f7-45bd-9d2a-13fca7c30ef7 node DatanodeRegistration(127.0.0.1:37011, datanodeUuid=c5b4826a-c83e-4331-8813-9b9896231180, infoPort=32797, infoSecurePort=0, ipcPort=39133, storageInfo=lv=-57;cid=testClusterID;nsid=1833191466;c=1693717196670), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-09-03 05:00:03,864 INFO [Listener at localhost/39133] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33813 2023-09-03 05:00:03,876 WARN [Listener at localhost/46829] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 05:00:03,904 WARN [Listener at localhost/46829] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-03 05:00:03,908 WARN [Listener at localhost/46829] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 05:00:03,909 INFO [Listener at localhost/46829] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 05:00:03,914 INFO [Listener at localhost/46829] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/java.io.tmpdir/Jetty_localhost_36161_datanode____.iyg27k/webapp 2023-09-03 05:00:04,004 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x8e77d6d57fae0ddc: Processing first storage report for DS-4099c782-1642-4c38-8d54-8cec20f5a12e from datanode 9eb1d0ba-8464-424c-8883-1673cdf5d6e6 2023-09-03 05:00:04,004 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x8e77d6d57fae0ddc: from storage DS-4099c782-1642-4c38-8d54-8cec20f5a12e node DatanodeRegistration(127.0.0.1:35551, datanodeUuid=9eb1d0ba-8464-424c-8883-1673cdf5d6e6, infoPort=37025, infoSecurePort=0, ipcPort=46829, storageInfo=lv=-57;cid=testClusterID;nsid=1833191466;c=1693717196670), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:00:04,005 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x8e77d6d57fae0ddc: Processing first storage report for DS-3e65fbc5-9e28-47a4-b4d3-2a5ffd546347 from datanode 9eb1d0ba-8464-424c-8883-1673cdf5d6e6 2023-09-03 05:00:04,005 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x8e77d6d57fae0ddc: from storage DS-3e65fbc5-9e28-47a4-b4d3-2a5ffd546347 node DatanodeRegistration(127.0.0.1:35551, datanodeUuid=9eb1d0ba-8464-424c-8883-1673cdf5d6e6, infoPort=37025, infoSecurePort=0, ipcPort=46829, storageInfo=lv=-57;cid=testClusterID;nsid=1833191466;c=1693717196670), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:00:04,051 INFO [Listener at localhost/46829] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36161 2023-09-03 05:00:04,077 WARN [Listener at localhost/33455] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 05:00:04,210 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x676588d98f3a4e2e: Processing first storage report for DS-3e289f15-0f2c-443b-a6eb-c04cc3e6e381 from datanode ad9eaa7d-d58f-4f42-959b-2bd1f84ac41f 2023-09-03 05:00:04,210 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x676588d98f3a4e2e: from storage DS-3e289f15-0f2c-443b-a6eb-c04cc3e6e381 node DatanodeRegistration(127.0.0.1:40899, datanodeUuid=ad9eaa7d-d58f-4f42-959b-2bd1f84ac41f, infoPort=35619, infoSecurePort=0, ipcPort=33455, storageInfo=lv=-57;cid=testClusterID;nsid=1833191466;c=1693717196670), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:00:04,210 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x676588d98f3a4e2e: Processing first storage report for DS-036bbba8-09a3-44e8-833f-2342685e7bc1 from datanode ad9eaa7d-d58f-4f42-959b-2bd1f84ac41f 2023-09-03 05:00:04,210 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x676588d98f3a4e2e: from storage DS-036bbba8-09a3-44e8-833f-2342685e7bc1 node DatanodeRegistration(127.0.0.1:40899, datanodeUuid=ad9eaa7d-d58f-4f42-959b-2bd1f84ac41f, infoPort=35619, infoSecurePort=0, ipcPort=33455, storageInfo=lv=-57;cid=testClusterID;nsid=1833191466;c=1693717196670), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:00:04,225 DEBUG [Listener at localhost/33455] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-09-03 05:00:04,228 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33586, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-09-03 05:00:04,238 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43827] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-09-03 05:00:04,238 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43827] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-09-03 05:00:04,242 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43827] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-03 05:00:04,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43827] procedure2.ProcedureExecutor(1028): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath 2023-09-03 05:00:04,247 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_PRE_OPERATION 2023-09-03 05:00:04,249 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-03 05:00:04,251 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43827] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRollOnDatanodeDeath" procId is: 9 2023-09-03 05:00:04,252 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/c6555a58050777e764b9d34419f8beba 2023-09-03 05:00:04,254 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/c6555a58050777e764b9d34419f8beba empty. 2023-09-03 05:00:04,256 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/c6555a58050777e764b9d34419f8beba 2023-09-03 05:00:04,256 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRollOnDatanodeDeath regions 2023-09-03 05:00:04,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43827] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-03 05:00:04,280 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/.tabledesc/.tableinfo.0000000001 2023-09-03 05:00:04,282 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(7675): creating {ENCODED => c6555a58050777e764b9d34419f8beba, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/.tmp 2023-09-03 05:00:04,299 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:04,299 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1604): Closing c6555a58050777e764b9d34419f8beba, disabling compactions & flushes 2023-09-03 05:00:04,299 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba. 2023-09-03 05:00:04,299 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba. 2023-09-03 05:00:04,299 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba. after waiting 0 ms 2023-09-03 05:00:04,299 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba. 2023-09-03 05:00:04,299 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba. 2023-09-03 05:00:04,299 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1558): Region close journal for c6555a58050777e764b9d34419f8beba: 2023-09-03 05:00:04,303 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ADD_TO_META 2023-09-03 05:00:04,305 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1693717204305"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693717204305"}]},"ts":"1693717204305"} 2023-09-03 05:00:04,308 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-03 05:00:04,309 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-03 05:00:04,310 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693717204309"}]},"ts":"1693717204309"} 2023-09-03 05:00:04,312 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLING in hbase:meta 2023-09-03 05:00:04,317 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=c6555a58050777e764b9d34419f8beba, ASSIGN}] 2023-09-03 05:00:04,319 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=c6555a58050777e764b9d34419f8beba, ASSIGN 2023-09-03 05:00:04,321 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=c6555a58050777e764b9d34419f8beba, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38963,1693717200832; forceNewPlan=false, retain=false 2023-09-03 05:00:04,472 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=c6555a58050777e764b9d34419f8beba, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38963,1693717200832 2023-09-03 05:00:04,472 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1693717204472"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693717204472"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693717204472"}]},"ts":"1693717204472"} 2023-09-03 05:00:04,476 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure c6555a58050777e764b9d34419f8beba, server=jenkins-hbase4.apache.org,38963,1693717200832}] 2023-09-03 05:00:04,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba. 2023-09-03 05:00:04,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c6555a58050777e764b9d34419f8beba, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba.', STARTKEY => '', ENDKEY => ''} 2023-09-03 05:00:04,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRollOnDatanodeDeath c6555a58050777e764b9d34419f8beba 2023-09-03 05:00:04,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:04,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c6555a58050777e764b9d34419f8beba 2023-09-03 05:00:04,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c6555a58050777e764b9d34419f8beba 2023-09-03 05:00:04,639 INFO [StoreOpener-c6555a58050777e764b9d34419f8beba-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region c6555a58050777e764b9d34419f8beba 2023-09-03 05:00:04,641 DEBUG [StoreOpener-c6555a58050777e764b9d34419f8beba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/default/TestLogRolling-testLogRollOnDatanodeDeath/c6555a58050777e764b9d34419f8beba/info 2023-09-03 05:00:04,641 DEBUG [StoreOpener-c6555a58050777e764b9d34419f8beba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/default/TestLogRolling-testLogRollOnDatanodeDeath/c6555a58050777e764b9d34419f8beba/info 2023-09-03 05:00:04,642 INFO [StoreOpener-c6555a58050777e764b9d34419f8beba-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c6555a58050777e764b9d34419f8beba columnFamilyName info 2023-09-03 05:00:04,642 INFO [StoreOpener-c6555a58050777e764b9d34419f8beba-1] regionserver.HStore(310): Store=c6555a58050777e764b9d34419f8beba/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:04,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/default/TestLogRolling-testLogRollOnDatanodeDeath/c6555a58050777e764b9d34419f8beba 2023-09-03 05:00:04,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/default/TestLogRolling-testLogRollOnDatanodeDeath/c6555a58050777e764b9d34419f8beba 2023-09-03 05:00:04,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c6555a58050777e764b9d34419f8beba 2023-09-03 05:00:04,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/default/TestLogRolling-testLogRollOnDatanodeDeath/c6555a58050777e764b9d34419f8beba/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-03 05:00:04,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c6555a58050777e764b9d34419f8beba; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=752800, jitterRate=-0.0427660197019577}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-03 05:00:04,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c6555a58050777e764b9d34419f8beba: 2023-09-03 05:00:04,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba., pid=11, masterSystemTime=1693717204628 2023-09-03 05:00:04,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba. 2023-09-03 05:00:04,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba. 2023-09-03 05:00:04,659 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=c6555a58050777e764b9d34419f8beba, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38963,1693717200832 2023-09-03 05:00:04,659 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1693717204658"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693717204658"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693717204658"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693717204658"}]},"ts":"1693717204658"} 2023-09-03 05:00:04,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=11, resume processing ppid=10 2023-09-03 05:00:04,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure c6555a58050777e764b9d34419f8beba, server=jenkins-hbase4.apache.org,38963,1693717200832 in 186 msec 2023-09-03 05:00:04,668 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=10, resume processing ppid=9 2023-09-03 05:00:04,669 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=c6555a58050777e764b9d34419f8beba, ASSIGN in 348 msec 2023-09-03 05:00:04,670 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-03 05:00:04,670 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693717204670"}]},"ts":"1693717204670"} 2023-09-03 05:00:04,672 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLED in hbase:meta 2023-09-03 05:00:04,675 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_POST_OPERATION 2023-09-03 05:00:04,677 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath in 433 msec 2023-09-03 05:00:07,982 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-03 05:00:08,114 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-09-03 05:00:08,116 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-09-03 05:00:08,116 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRollOnDatanodeDeath' 2023-09-03 05:00:09,824 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-03 05:00:09,825 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-09-03 05:00:14,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43827] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-03 05:00:14,273 INFO [Listener at localhost/33455] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRollOnDatanodeDeath, procId: 9 completed 2023-09-03 05:00:14,273 DEBUG [Listener at localhost/33455] hbase.HBaseTestingUtility(3430): Waiting until all regions of table TestLogRolling-testLogRollOnDatanodeDeath get assigned. Timeout = 60000ms 2023-09-03 05:00:14,275 INFO [Listener at localhost/33455] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-09-03 05:00:14,280 INFO [Listener at localhost/33455] hbase.HBaseTestingUtility(3484): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned to meta. Checking AM states. 2023-09-03 05:00:14,281 INFO [Listener at localhost/33455] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-09-03 05:00:14,281 INFO [Listener at localhost/33455] hbase.HBaseTestingUtility(3504): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned. 2023-09-03 05:00:14,293 DEBUG [Listener at localhost/33455] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRollOnDatanodeDeath 2023-09-03 05:00:14,294 DEBUG [Listener at localhost/33455] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba. 2023-09-03 05:00:14,295 WARN [Listener at localhost/33455] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-03 05:00:14,304 INFO [Listener at localhost/33455] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:00:14,306 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_1136839929_17 at /127.0.0.1:56206 [Receiving block BP-1084565183-172.31.14.131-1693717196670:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:45159:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:56206 dst: /127.0.0.1:45159 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:14,306 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_1136839929_17 at /127.0.0.1:56196 [Receiving block BP-1084565183-172.31.14.131-1693717196670:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:45159:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:56196 dst: /127.0.0.1:45159 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:14,306 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1696673971_17 at /127.0.0.1:56188 [Receiving block BP-1084565183-172.31.14.131-1693717196670:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:45159:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:56188 dst: /127.0.0.1:45159 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:14,407 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1696673971_17 at /127.0.0.1:60724 [Receiving block BP-1084565183-172.31.14.131-1693717196670:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:40809:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:60724 dst: /127.0.0.1:40809 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:14,408 WARN [BP-1084565183-172.31.14.131-1693717196670 heartbeating to localhost/127.0.0.1:39237] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-03 05:00:14,408 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_1136839929_17 at /127.0.0.1:60734 [Receiving block BP-1084565183-172.31.14.131-1693717196670:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:40809:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:60734 dst: /127.0.0.1:40809 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:14,407 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_1136839929_17 at /127.0.0.1:60738 [Receiving block BP-1084565183-172.31.14.131-1693717196670:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:40809:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:60738 dst: /127.0.0.1:40809 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:14,409 WARN [BP-1084565183-172.31.14.131-1693717196670 heartbeating to localhost/127.0.0.1:39237] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1084565183-172.31.14.131-1693717196670 (Datanode Uuid 159899f9-8186-4c2d-ad8e-cecc5127279c) service to localhost/127.0.0.1:39237 2023-09-03 05:00:14,411 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5/dfs/data/data1/current/BP-1084565183-172.31.14.131-1693717196670] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:00:14,412 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5/dfs/data/data2/current/BP-1084565183-172.31.14.131-1693717196670] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:00:14,420 WARN [Listener at localhost/33455] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-03 05:00:14,422 WARN [Listener at localhost/33455] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 05:00:14,424 INFO [Listener at localhost/33455] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 05:00:14,428 INFO [Listener at localhost/33455] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/java.io.tmpdir/Jetty_localhost_43939_datanode____1dsgo7/webapp 2023-09-03 05:00:14,524 INFO [Listener at localhost/33455] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43939 2023-09-03 05:00:14,533 WARN [Listener at localhost/36445] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 05:00:14,542 WARN [AsyncFSWAL-0-hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610-prefix:jenkins-hbase4.apache.org,38963,1693717200832] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:14,544 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C38963%2C1693717200832:(num 1693717202319) roll requested 2023-09-03 05:00:14,562 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45159,DS-ea21ea74-aa1b-49ab-954b-fbb16cc8a608,DISK] 2023-09-03 05:00:14,562 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37011,DS-a31c2e55-d189-405f-8eed-6648ac367e8a,DISK] 2023-09-03 05:00:14,567 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.1693717202319 with entries=6, filesize=3.19 KB; new WAL /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.1693717214544 2023-09-03 05:00:14,568 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45159,DS-ea21ea74-aa1b-49ab-954b-fbb16cc8a608,DISK], DatanodeInfoWithStorage[127.0.0.1:37011,DS-a31c2e55-d189-405f-8eed-6648ac367e8a,DISK]] 2023-09-03 05:00:14,568 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.1693717202319 is not closed yet, will try archiving it next time 2023-09-03 05:00:14,569 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-09-03 05:00:14,570 INFO [Listener at localhost/36445] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-09-03 05:00:14,571 DEBUG [Listener at localhost/36445] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x78deaf86 to 127.0.0.1:51414 2023-09-03 05:00:14,571 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:14,571 DEBUG [Listener at localhost/36445] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:00:14,571 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:14,572 DEBUG [Listener at localhost/36445] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-09-03 05:00:14,572 DEBUG [Listener at localhost/36445] util.JVMClusterUtil(257): Found active master hash=1453410060, stopped=false 2023-09-03 05:00:14,572 INFO [Listener at localhost/36445] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,43827,1693717199663 2023-09-03 05:00:14,574 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.1693717202319 2023-09-03 05:00:14,574 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-03 05:00:14,574 INFO [Listener at localhost/36445] procedure2.ProcedureExecutor(628): Stopping 2023-09-03 05:00:14,574 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:14,574 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): regionserver:38963-0x1007028af780001, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-03 05:00:14,575 DEBUG [Listener at localhost/36445] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x73eb9cf3 to 127.0.0.1:51414 2023-09-03 05:00:14,576 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:00:14,576 DEBUG [Listener at localhost/36445] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:00:14,577 INFO [Listener at localhost/36445] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38963,1693717200832' ***** 2023-09-03 05:00:14,577 INFO [Listener at localhost/36445] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-09-03 05:00:14,577 INFO [RS:0;jenkins-hbase4:38963] regionserver.HeapMemoryManager(220): Stopping 2023-09-03 05:00:14,577 INFO [RS:0;jenkins-hbase4:38963] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-09-03 05:00:14,577 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-09-03 05:00:14,577 INFO [RS:0;jenkins-hbase4:38963] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-09-03 05:00:14,578 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38963-0x1007028af780001, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:00:14,578 INFO [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(3305): Received CLOSE for c6555a58050777e764b9d34419f8beba 2023-09-03 05:00:14,579 INFO [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(3305): Received CLOSE for 64ccc3a1ad492104480fb5b883ad071e 2023-09-03 05:00:14,580 INFO [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38963,1693717200832 2023-09-03 05:00:14,580 DEBUG [RS:0;jenkins-hbase4:38963] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x669fb05d to 127.0.0.1:51414 2023-09-03 05:00:14,580 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c6555a58050777e764b9d34419f8beba, disabling compactions & flushes 2023-09-03 05:00:14,580 DEBUG [RS:0;jenkins-hbase4:38963] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:00:14,580 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba. 2023-09-03 05:00:14,580 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba. 2023-09-03 05:00:14,580 INFO [RS:0;jenkins-hbase4:38963] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-09-03 05:00:14,580 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba. after waiting 0 ms 2023-09-03 05:00:14,581 INFO [RS:0;jenkins-hbase4:38963] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-09-03 05:00:14,581 INFO [RS:0;jenkins-hbase4:38963] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-09-03 05:00:14,581 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba. 2023-09-03 05:00:14,581 INFO [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-09-03 05:00:14,581 WARN [IPC Server handler 4 on default port 39237] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.1693717202319 has not been closed. Lease recovery is in progress. RecoveryId = 1015 for block blk_1073741832_1008 2023-09-03 05:00:14,581 INFO [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-09-03 05:00:14,582 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-03 05:00:14,582 DEBUG [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(1478): Online Regions={c6555a58050777e764b9d34419f8beba=TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba., 64ccc3a1ad492104480fb5b883ad071e=hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e., 1588230740=hbase:meta,,1.1588230740} 2023-09-03 05:00:14,582 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-03 05:00:14,582 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-03 05:00:14,582 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing c6555a58050777e764b9d34419f8beba 1/1 column families, dataSize=2.05 KB heapSize=2.45 KB 2023-09-03 05:00:14,582 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-03 05:00:14,582 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-03 05:00:14,582 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.92 KB heapSize=5.45 KB 2023-09-03 05:00:14,584 DEBUG [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(1504): Waiting on 1588230740, 64ccc3a1ad492104480fb5b883ad071e, c6555a58050777e764b9d34419f8beba 2023-09-03 05:00:14,584 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.1693717202319 after 10ms 2023-09-03 05:00:14,592 WARN [AsyncFSWAL-0-hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610-prefix:jenkins-hbase4.apache.org,38963,1693717200832.meta] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:14,593 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C38963%2C1693717200832.meta:.meta(num 1693717202407) roll requested 2023-09-03 05:00:14,611 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45159,DS-ea21ea74-aa1b-49ab-954b-fbb16cc8a608,DISK] 2023-09-03 05:00:14,611 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37011,DS-a31c2e55-d189-405f-8eed-6648ac367e8a,DISK] 2023-09-03 05:00:14,621 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.meta.1693717202407.meta with entries=12, filesize=3.87 KB; new WAL /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.meta.1693717214593.meta 2023-09-03 05:00:14,621 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45159,DS-ea21ea74-aa1b-49ab-954b-fbb16cc8a608,DISK], DatanodeInfoWithStorage[127.0.0.1:37011,DS-a31c2e55-d189-405f-8eed-6648ac367e8a,DISK]] 2023-09-03 05:00:14,621 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.meta.1693717202407.meta is not closed yet, will try archiving it next time 2023-09-03 05:00:14,621 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:14,630 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:14,631 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.meta.1693717202407.meta 2023-09-03 05:00:14,632 WARN [IPC Server handler 2 on default port 39237] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.meta.1693717202407.meta has not been closed. Lease recovery is in progress. RecoveryId = 1017 for block blk_1073741833_1009 2023-09-03 05:00:14,632 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.meta.1693717202407.meta after 1ms 2023-09-03 05:00:14,679 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.69 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/.tmp/info/cf7e94882f724b52bdf61d55d97b13e5 2023-09-03 05:00:14,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.05 KB at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/default/TestLogRolling-testLogRollOnDatanodeDeath/c6555a58050777e764b9d34419f8beba/.tmp/info/03591fec82cd4ebaadeae53ba1916aba 2023-09-03 05:00:14,687 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd15b037c19e272c1: Processing first storage report for DS-7d16f019-67b5-49b5-b8d9-77cd2d44437e from datanode 159899f9-8186-4c2d-ad8e-cecc5127279c 2023-09-03 05:00:14,688 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd15b037c19e272c1: from storage DS-7d16f019-67b5-49b5-b8d9-77cd2d44437e node DatanodeRegistration(127.0.0.1:36721, datanodeUuid=159899f9-8186-4c2d-ad8e-cecc5127279c, infoPort=39209, infoSecurePort=0, ipcPort=36445, storageInfo=lv=-57;cid=testClusterID;nsid=1833191466;c=1693717196670), blocks: 6, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-09-03 05:00:14,688 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd15b037c19e272c1: Processing first storage report for DS-0a97c607-b024-4448-920a-13176a55ff03 from datanode 159899f9-8186-4c2d-ad8e-cecc5127279c 2023-09-03 05:00:14,689 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd15b037c19e272c1: from storage DS-0a97c607-b024-4448-920a-13176a55ff03 node DatanodeRegistration(127.0.0.1:36721, datanodeUuid=159899f9-8186-4c2d-ad8e-cecc5127279c, infoPort=39209, infoSecurePort=0, ipcPort=36445, storageInfo=lv=-57;cid=testClusterID;nsid=1833191466;c=1693717196670), blocks: 6, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:00:14,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/default/TestLogRolling-testLogRollOnDatanodeDeath/c6555a58050777e764b9d34419f8beba/.tmp/info/03591fec82cd4ebaadeae53ba1916aba as hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/default/TestLogRolling-testLogRollOnDatanodeDeath/c6555a58050777e764b9d34419f8beba/info/03591fec82cd4ebaadeae53ba1916aba 2023-09-03 05:00:14,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/default/TestLogRolling-testLogRollOnDatanodeDeath/c6555a58050777e764b9d34419f8beba/info/03591fec82cd4ebaadeae53ba1916aba, entries=2, sequenceid=6, filesize=6.8 K 2023-09-03 05:00:14,750 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/.tmp/table/e14d350e17c7426f861b4ead8db5d69e 2023-09-03 05:00:14,751 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.05 KB/2102, heapSize ~2.44 KB/2496, currentSize=0 B/0 for c6555a58050777e764b9d34419f8beba in 169ms, sequenceid=6, compaction requested=false 2023-09-03 05:00:14,760 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/default/TestLogRolling-testLogRollOnDatanodeDeath/c6555a58050777e764b9d34419f8beba/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-09-03 05:00:14,761 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba. 2023-09-03 05:00:14,762 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c6555a58050777e764b9d34419f8beba: 2023-09-03 05:00:14,762 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1693717204238.c6555a58050777e764b9d34419f8beba. 2023-09-03 05:00:14,762 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/.tmp/info/cf7e94882f724b52bdf61d55d97b13e5 as hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/info/cf7e94882f724b52bdf61d55d97b13e5 2023-09-03 05:00:14,762 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 64ccc3a1ad492104480fb5b883ad071e, disabling compactions & flushes 2023-09-03 05:00:14,763 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e. 2023-09-03 05:00:14,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e. 2023-09-03 05:00:14,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e. after waiting 0 ms 2023-09-03 05:00:14,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e. 2023-09-03 05:00:14,763 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 64ccc3a1ad492104480fb5b883ad071e 1/1 column families, dataSize=78 B heapSize=488 B 2023-09-03 05:00:14,772 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/info/cf7e94882f724b52bdf61d55d97b13e5, entries=20, sequenceid=14, filesize=7.4 K 2023-09-03 05:00:14,775 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/.tmp/table/e14d350e17c7426f861b4ead8db5d69e as hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/table/e14d350e17c7426f861b4ead8db5d69e 2023-09-03 05:00:14,784 DEBUG [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(1504): Waiting on 1588230740, 64ccc3a1ad492104480fb5b883ad071e 2023-09-03 05:00:14,787 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/table/e14d350e17c7426f861b4ead8db5d69e, entries=4, sequenceid=14, filesize=4.8 K 2023-09-03 05:00:14,788 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.92 KB/2994, heapSize ~5.16 KB/5288, currentSize=0 B/0 for 1588230740 in 206ms, sequenceid=14, compaction requested=false 2023-09-03 05:00:14,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/namespace/64ccc3a1ad492104480fb5b883ad071e/.tmp/info/bae97feac5c244e6bec6c7dcd1f7db52 2023-09-03 05:00:14,797 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-09-03 05:00:14,798 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-03 05:00:14,799 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-03 05:00:14,799 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-03 05:00:14,799 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-09-03 05:00:14,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/namespace/64ccc3a1ad492104480fb5b883ad071e/.tmp/info/bae97feac5c244e6bec6c7dcd1f7db52 as hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/namespace/64ccc3a1ad492104480fb5b883ad071e/info/bae97feac5c244e6bec6c7dcd1f7db52 2023-09-03 05:00:14,809 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/namespace/64ccc3a1ad492104480fb5b883ad071e/info/bae97feac5c244e6bec6c7dcd1f7db52, entries=2, sequenceid=6, filesize=4.8 K 2023-09-03 05:00:14,810 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 64ccc3a1ad492104480fb5b883ad071e in 47ms, sequenceid=6, compaction requested=false 2023-09-03 05:00:14,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/data/hbase/namespace/64ccc3a1ad492104480fb5b883ad071e/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-09-03 05:00:14,820 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e. 2023-09-03 05:00:14,820 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 64ccc3a1ad492104480fb5b883ad071e: 2023-09-03 05:00:14,820 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693717202682.64ccc3a1ad492104480fb5b883ad071e. 2023-09-03 05:00:14,984 INFO [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38963,1693717200832; all regions closed. 2023-09-03 05:00:15,161 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-09-03 05:00:15,161 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-09-03 05:00:16,158 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-09-03 05:00:18,585 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.1693717202319 after 4011ms 2023-09-03 05:00:18,586 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.1693717202319 to hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/oldWALs/jenkins-hbase4.apache.org%2C38963%2C1693717200832.1693717202319 2023-09-03 05:00:18,633 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.meta.1693717202407.meta after 4002ms 2023-09-03 05:00:18,634 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/WALs/jenkins-hbase4.apache.org,38963,1693717200832/jenkins-hbase4.apache.org%2C38963%2C1693717200832.meta.1693717202407.meta to hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/oldWALs/jenkins-hbase4.apache.org%2C38963%2C1693717200832.meta.1693717202407.meta 2023-09-03 05:00:18,639 DEBUG [RS:0;jenkins-hbase4:38963] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/oldWALs 2023-09-03 05:00:18,639 INFO [RS:0;jenkins-hbase4:38963] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38963%2C1693717200832.meta:.meta(num 1693717214593) 2023-09-03 05:00:18,647 DEBUG [RS:0;jenkins-hbase4:38963] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/oldWALs 2023-09-03 05:00:18,647 INFO [RS:0;jenkins-hbase4:38963] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38963%2C1693717200832:(num 1693717214544) 2023-09-03 05:00:18,647 DEBUG [RS:0;jenkins-hbase4:38963] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:00:18,647 INFO [RS:0;jenkins-hbase4:38963] regionserver.LeaseManager(133): Closed leases 2023-09-03 05:00:18,647 INFO [RS:0;jenkins-hbase4:38963] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-09-03 05:00:18,648 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-03 05:00:18,648 INFO [RS:0;jenkins-hbase4:38963] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38963 2023-09-03 05:00:18,655 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): regionserver:38963-0x1007028af780001, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38963,1693717200832 2023-09-03 05:00:18,655 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-03 05:00:18,655 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): regionserver:38963-0x1007028af780001, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-03 05:00:18,656 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38963,1693717200832] 2023-09-03 05:00:18,657 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38963,1693717200832; numProcessing=1 2023-09-03 05:00:18,659 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,38963,1693717200832 already deleted, retry=false 2023-09-03 05:00:18,659 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38963,1693717200832 expired; onlineServers=0 2023-09-03 05:00:18,660 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43827,1693717199663' ***** 2023-09-03 05:00:18,660 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-09-03 05:00:18,660 DEBUG [M:0;jenkins-hbase4:43827] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@179c4d37, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-03 05:00:18,660 INFO [M:0;jenkins-hbase4:43827] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43827,1693717199663 2023-09-03 05:00:18,660 INFO [M:0;jenkins-hbase4:43827] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43827,1693717199663; all regions closed. 2023-09-03 05:00:18,660 DEBUG [M:0;jenkins-hbase4:43827] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:00:18,660 DEBUG [M:0;jenkins-hbase4:43827] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-09-03 05:00:18,660 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-09-03 05:00:18,661 DEBUG [M:0;jenkins-hbase4:43827] cleaner.HFileCleaner(317): Stopping file delete threads 2023-09-03 05:00:18,661 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693717201923] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693717201923,5,FailOnTimeoutGroup] 2023-09-03 05:00:18,662 INFO [M:0;jenkins-hbase4:43827] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-09-03 05:00:18,661 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693717201924] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693717201924,5,FailOnTimeoutGroup] 2023-09-03 05:00:18,662 INFO [M:0;jenkins-hbase4:43827] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-09-03 05:00:18,663 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-09-03 05:00:18,663 INFO [M:0;jenkins-hbase4:43827] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-09-03 05:00:18,663 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:18,663 DEBUG [M:0;jenkins-hbase4:43827] master.HMaster(1512): Stopping service threads 2023-09-03 05:00:18,664 INFO [M:0;jenkins-hbase4:43827] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-09-03 05:00:18,664 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-03 05:00:18,664 ERROR [M:0;jenkins-hbase4:43827] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] 2023-09-03 05:00:18,665 INFO [M:0;jenkins-hbase4:43827] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-09-03 05:00:18,665 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-09-03 05:00:18,665 DEBUG [M:0;jenkins-hbase4:43827] zookeeper.ZKUtil(398): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-09-03 05:00:18,665 WARN [M:0;jenkins-hbase4:43827] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-09-03 05:00:18,666 INFO [M:0;jenkins-hbase4:43827] assignment.AssignmentManager(315): Stopping assignment manager 2023-09-03 05:00:18,666 INFO [M:0;jenkins-hbase4:43827] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-09-03 05:00:18,666 DEBUG [M:0;jenkins-hbase4:43827] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-03 05:00:18,666 INFO [M:0;jenkins-hbase4:43827] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:00:18,666 DEBUG [M:0;jenkins-hbase4:43827] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:00:18,666 DEBUG [M:0;jenkins-hbase4:43827] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-03 05:00:18,666 DEBUG [M:0;jenkins-hbase4:43827] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:00:18,667 INFO [M:0;jenkins-hbase4:43827] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.08 KB heapSize=45.73 KB 2023-09-03 05:00:18,667 WARN [AsyncFSWAL-0-hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData-prefix:jenkins-hbase4.apache.org,43827,1693717199663] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:18,668 DEBUG [master:store-WAL-Roller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C43827%2C1693717199663:(num 1693717201258) roll requested 2023-09-03 05:00:18,686 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45159,DS-ea21ea74-aa1b-49ab-954b-fbb16cc8a608,DISK] 2023-09-03 05:00:18,686 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35551,DS-4099c782-1642-4c38-8d54-8cec20f5a12e,DISK] 2023-09-03 05:00:18,689 INFO [master:store-WAL-Roller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/WALs/jenkins-hbase4.apache.org,43827,1693717199663/jenkins-hbase4.apache.org%2C43827%2C1693717199663.1693717201258 with entries=89, filesize=43.93 KB; new WAL /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/WALs/jenkins-hbase4.apache.org,43827,1693717199663/jenkins-hbase4.apache.org%2C43827%2C1693717199663.1693717218668 2023-09-03 05:00:18,690 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45159,DS-ea21ea74-aa1b-49ab-954b-fbb16cc8a608,DISK], DatanodeInfoWithStorage[127.0.0.1:35551,DS-4099c782-1642-4c38-8d54-8cec20f5a12e,DISK]] 2023-09-03 05:00:18,690 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(716): hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/WALs/jenkins-hbase4.apache.org,43827,1693717199663/jenkins-hbase4.apache.org%2C43827%2C1693717199663.1693717201258 is not closed yet, will try archiving it next time 2023-09-03 05:00:18,690 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:18,690 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-03 05:00:18,690 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/WALs/jenkins-hbase4.apache.org,43827,1693717199663/jenkins-hbase4.apache.org%2C43827%2C1693717199663.1693717201258 2023-09-03 05:00:18,691 WARN [IPC Server handler 2 on default port 39237] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/WALs/jenkins-hbase4.apache.org,43827,1693717199663/jenkins-hbase4.apache.org%2C43827%2C1693717199663.1693717201258 has not been closed. Lease recovery is in progress. RecoveryId = 1023 for block blk_1073741829_1005 2023-09-03 05:00:18,692 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/WALs/jenkins-hbase4.apache.org,43827,1693717199663/jenkins-hbase4.apache.org%2C43827%2C1693717199663.1693717201258 after 2ms 2023-09-03 05:00:18,705 INFO [M:0;jenkins-hbase4:43827] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.08 KB at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/c16daed963cf46c0bf37cf33e75552fd 2023-09-03 05:00:18,713 DEBUG [M:0;jenkins-hbase4:43827] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/c16daed963cf46c0bf37cf33e75552fd as hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/c16daed963cf46c0bf37cf33e75552fd 2023-09-03 05:00:18,720 INFO [M:0;jenkins-hbase4:43827] regionserver.HStore(1080): Added hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/c16daed963cf46c0bf37cf33e75552fd, entries=11, sequenceid=91, filesize=7.0 K 2023-09-03 05:00:18,721 INFO [M:0;jenkins-hbase4:43827] regionserver.HRegion(2948): Finished flush of dataSize ~38.08 KB/38997, heapSize ~45.72 KB/46816, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 55ms, sequenceid=91, compaction requested=false 2023-09-03 05:00:18,722 INFO [M:0;jenkins-hbase4:43827] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:00:18,722 DEBUG [M:0;jenkins-hbase4:43827] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-03 05:00:18,756 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): regionserver:38963-0x1007028af780001, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:00:18,757 INFO [RS:0;jenkins-hbase4:38963] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38963,1693717200832; zookeeper connection closed. 2023-09-03 05:00:18,757 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): regionserver:38963-0x1007028af780001, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:00:18,757 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2b2f7acc] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2b2f7acc 2023-09-03 05:00:18,757 INFO [Listener at localhost/36445] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-09-03 05:00:20,310 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-03 05:00:20,348 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-03 05:00:22,693 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/WALs/jenkins-hbase4.apache.org,43827,1693717199663/jenkins-hbase4.apache.org%2C43827%2C1693717199663.1693717201258 after 4003ms 2023-09-03 05:00:22,693 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/WALs/jenkins-hbase4.apache.org,43827,1693717199663/jenkins-hbase4.apache.org%2C43827%2C1693717199663.1693717201258 to hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/oldWALs/jenkins-hbase4.apache.org%2C43827%2C1693717199663.1693717201258 2023-09-03 05:00:22,699 INFO [WAL-Archive-0] region.MasterRegionUtils(50): Moved hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/MasterData/oldWALs/jenkins-hbase4.apache.org%2C43827%2C1693717199663.1693717201258 to hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/oldWALs/jenkins-hbase4.apache.org%2C43827%2C1693717199663.1693717201258$masterlocalwal$ 2023-09-03 05:00:22,699 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-03 05:00:22,700 INFO [M:0;jenkins-hbase4:43827] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-09-03 05:00:22,700 INFO [M:0;jenkins-hbase4:43827] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43827 2023-09-03 05:00:22,703 DEBUG [M:0;jenkins-hbase4:43827] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,43827,1693717199663 already deleted, retry=false 2023-09-03 05:00:22,804 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:00:22,805 INFO [M:0;jenkins-hbase4:43827] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43827,1693717199663; zookeeper connection closed. 2023-09-03 05:00:22,805 DEBUG [Listener at localhost/39895-EventThread] zookeeper.ZKWatcher(600): master:43827-0x1007028af780000, quorum=127.0.0.1:51414, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:00:22,807 WARN [Listener at localhost/36445] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-03 05:00:22,811 INFO [Listener at localhost/36445] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:00:22,917 WARN [BP-1084565183-172.31.14.131-1693717196670 heartbeating to localhost/127.0.0.1:39237] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-03 05:00:22,917 WARN [BP-1084565183-172.31.14.131-1693717196670 heartbeating to localhost/127.0.0.1:39237] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1084565183-172.31.14.131-1693717196670 (Datanode Uuid 159899f9-8186-4c2d-ad8e-cecc5127279c) service to localhost/127.0.0.1:39237 2023-09-03 05:00:22,918 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5/dfs/data/data1/current/BP-1084565183-172.31.14.131-1693717196670] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:00:22,918 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5/dfs/data/data2/current/BP-1084565183-172.31.14.131-1693717196670] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:00:22,921 WARN [Listener at localhost/36445] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-03 05:00:22,924 INFO [Listener at localhost/36445] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:00:23,027 WARN [BP-1084565183-172.31.14.131-1693717196670 heartbeating to localhost/127.0.0.1:39237] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-03 05:00:23,027 WARN [BP-1084565183-172.31.14.131-1693717196670 heartbeating to localhost/127.0.0.1:39237] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1084565183-172.31.14.131-1693717196670 (Datanode Uuid ad9eaa7d-d58f-4f42-959b-2bd1f84ac41f) service to localhost/127.0.0.1:39237 2023-09-03 05:00:23,028 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5/dfs/data/data9/current/BP-1084565183-172.31.14.131-1693717196670] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:00:23,028 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5/dfs/data/data10/current/BP-1084565183-172.31.14.131-1693717196670] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:00:23,030 WARN [Listener at localhost/36445] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-03 05:00:23,033 INFO [Listener at localhost/36445] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:00:23,136 WARN [BP-1084565183-172.31.14.131-1693717196670 heartbeating to localhost/127.0.0.1:39237] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-03 05:00:23,137 WARN [BP-1084565183-172.31.14.131-1693717196670 heartbeating to localhost/127.0.0.1:39237] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1084565183-172.31.14.131-1693717196670 (Datanode Uuid 9eb1d0ba-8464-424c-8883-1673cdf5d6e6) service to localhost/127.0.0.1:39237 2023-09-03 05:00:23,137 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5/dfs/data/data7/current/BP-1084565183-172.31.14.131-1693717196670] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:00:23,138 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5/dfs/data/data8/current/BP-1084565183-172.31.14.131-1693717196670] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:00:23,139 WARN [Listener at localhost/36445] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-03 05:00:23,142 INFO [Listener at localhost/36445] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:00:23,245 WARN [BP-1084565183-172.31.14.131-1693717196670 heartbeating to localhost/127.0.0.1:39237] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-03 05:00:23,245 WARN [BP-1084565183-172.31.14.131-1693717196670 heartbeating to localhost/127.0.0.1:39237] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1084565183-172.31.14.131-1693717196670 (Datanode Uuid c5b4826a-c83e-4331-8813-9b9896231180) service to localhost/127.0.0.1:39237 2023-09-03 05:00:23,246 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5/dfs/data/data5/current/BP-1084565183-172.31.14.131-1693717196670] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:00:23,246 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5/dfs/data/data6/current/BP-1084565183-172.31.14.131-1693717196670] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:00:23,247 WARN [Listener at localhost/36445] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-03 05:00:23,249 INFO [Listener at localhost/36445] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:00:23,351 WARN [BP-1084565183-172.31.14.131-1693717196670 heartbeating to localhost/127.0.0.1:39237] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-03 05:00:23,351 WARN [BP-1084565183-172.31.14.131-1693717196670 heartbeating to localhost/127.0.0.1:39237] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1084565183-172.31.14.131-1693717196670 (Datanode Uuid 36e45ea4-2bce-487a-965e-197048fe474c) service to localhost/127.0.0.1:39237 2023-09-03 05:00:23,351 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5/dfs/data/data3/current/BP-1084565183-172.31.14.131-1693717196670] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:00:23,352 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/cluster_5391e4cd-23df-db7a-43ec-47b3a6dca1f5/dfs/data/data4/current/BP-1084565183-172.31.14.131-1693717196670] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:00:23,385 INFO [Listener at localhost/36445] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:00:23,500 INFO [Listener at localhost/36445] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-09-03 05:00:23,555 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-09-03 05:00:23,570 INFO [Listener at localhost/36445] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=78 (was 10) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/36445 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: ForkJoinPool-2-worker-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: IPC Client (1031196848) connection to localhost/127.0.0.1:39237 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:39237 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:39237 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1031196848) connection to localhost/127.0.0.1:39237 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: ForkJoinPool-2-worker-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (1031196848) connection to localhost/127.0.0.1:39237 from jenkins.hfs.0 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@2cf44a2 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=467 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=198 (was 189) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=8624 (was 9345) 2023-09-03 05:00:23,583 INFO [Listener at localhost/36445] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=78, OpenFileDescriptor=467, MaxFileDescriptor=60000, SystemLoadAverage=198, ProcessCount=167, AvailableMemoryMB=8624 2023-09-03 05:00:23,583 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-09-03 05:00:23,584 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/hadoop.log.dir so I do NOT create it in target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0 2023-09-03 05:00:23,584 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b78eb0c3-9250-d028-2c21-874cc3ee7082/hadoop.tmp.dir so I do NOT create it in target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0 2023-09-03 05:00:23,584 INFO [Listener at localhost/36445] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/cluster_fce263f4-3716-36b0-1730-b29470dde46f, deleteOnExit=true 2023-09-03 05:00:23,584 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-09-03 05:00:23,584 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/test.cache.data in system properties and HBase conf 2023-09-03 05:00:23,584 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/hadoop.tmp.dir in system properties and HBase conf 2023-09-03 05:00:23,584 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/hadoop.log.dir in system properties and HBase conf 2023-09-03 05:00:23,584 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/mapreduce.cluster.local.dir in system properties and HBase conf 2023-09-03 05:00:23,585 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-09-03 05:00:23,585 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-09-03 05:00:23,585 DEBUG [Listener at localhost/36445] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-09-03 05:00:23,585 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-09-03 05:00:23,585 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-09-03 05:00:23,586 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-09-03 05:00:23,586 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-03 05:00:23,586 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-09-03 05:00:23,586 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-09-03 05:00:23,587 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-03 05:00:23,587 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-03 05:00:23,587 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-09-03 05:00:23,587 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/nfs.dump.dir in system properties and HBase conf 2023-09-03 05:00:23,587 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/java.io.tmpdir in system properties and HBase conf 2023-09-03 05:00:23,588 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-03 05:00:23,588 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-09-03 05:00:23,588 INFO [Listener at localhost/36445] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-09-03 05:00:23,593 WARN [Listener at localhost/36445] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-03 05:00:23,593 WARN [Listener at localhost/36445] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-03 05:00:23,649 WARN [Listener at localhost/36445] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 05:00:23,652 INFO [Listener at localhost/36445] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 05:00:23,659 INFO [Listener at localhost/36445] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/java.io.tmpdir/Jetty_localhost_43685_hdfs____.ioqj5u/webapp 2023-09-03 05:00:23,752 INFO [Listener at localhost/36445] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43685 2023-09-03 05:00:23,756 WARN [Listener at localhost/36445] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-03 05:00:23,756 WARN [Listener at localhost/36445] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-03 05:00:23,799 WARN [Listener at localhost/45657] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 05:00:23,810 WARN [Listener at localhost/45657] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-03 05:00:23,813 WARN [Listener at localhost/45657] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 05:00:23,814 INFO [Listener at localhost/45657] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 05:00:23,819 INFO [Listener at localhost/45657] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/java.io.tmpdir/Jetty_localhost_42787_datanode____.13v6qp/webapp 2023-09-03 05:00:23,912 INFO [Listener at localhost/45657] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42787 2023-09-03 05:00:23,918 WARN [Listener at localhost/37835] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 05:00:23,931 WARN [Listener at localhost/37835] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-03 05:00:23,933 WARN [Listener at localhost/37835] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 05:00:23,934 INFO [Listener at localhost/37835] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 05:00:23,939 INFO [Listener at localhost/37835] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/java.io.tmpdir/Jetty_localhost_43543_datanode____.6k7rb8/webapp 2023-09-03 05:00:24,026 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x2ff4b1d47c94c99a: Processing first storage report for DS-d7d9187f-2c3c-4690-b735-31baabf2e20c from datanode 5fa208d1-5240-457d-b878-1155bdc696fb 2023-09-03 05:00:24,027 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x2ff4b1d47c94c99a: from storage DS-d7d9187f-2c3c-4690-b735-31baabf2e20c node DatanodeRegistration(127.0.0.1:34017, datanodeUuid=5fa208d1-5240-457d-b878-1155bdc696fb, infoPort=45991, infoSecurePort=0, ipcPort=37835, storageInfo=lv=-57;cid=testClusterID;nsid=1611104851;c=1693717223596), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:00:24,027 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x2ff4b1d47c94c99a: Processing first storage report for DS-def102a6-cd63-44e6-9e4c-ec2ed9f25495 from datanode 5fa208d1-5240-457d-b878-1155bdc696fb 2023-09-03 05:00:24,027 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x2ff4b1d47c94c99a: from storage DS-def102a6-cd63-44e6-9e4c-ec2ed9f25495 node DatanodeRegistration(127.0.0.1:34017, datanodeUuid=5fa208d1-5240-457d-b878-1155bdc696fb, infoPort=45991, infoSecurePort=0, ipcPort=37835, storageInfo=lv=-57;cid=testClusterID;nsid=1611104851;c=1693717223596), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:00:24,042 INFO [Listener at localhost/37835] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43543 2023-09-03 05:00:24,050 WARN [Listener at localhost/40333] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 05:00:24,147 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe454326cc7e49201: Processing first storage report for DS-82878c9e-a256-484e-8cac-a79479bf91fd from datanode 9eeecc76-a237-4311-bde6-01c24676dac7 2023-09-03 05:00:24,147 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe454326cc7e49201: from storage DS-82878c9e-a256-484e-8cac-a79479bf91fd node DatanodeRegistration(127.0.0.1:34525, datanodeUuid=9eeecc76-a237-4311-bde6-01c24676dac7, infoPort=43265, infoSecurePort=0, ipcPort=40333, storageInfo=lv=-57;cid=testClusterID;nsid=1611104851;c=1693717223596), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:00:24,147 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe454326cc7e49201: Processing first storage report for DS-04c5a769-68d9-46bd-8472-789855d7a5a6 from datanode 9eeecc76-a237-4311-bde6-01c24676dac7 2023-09-03 05:00:24,148 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe454326cc7e49201: from storage DS-04c5a769-68d9-46bd-8472-789855d7a5a6 node DatanodeRegistration(127.0.0.1:34525, datanodeUuid=9eeecc76-a237-4311-bde6-01c24676dac7, infoPort=43265, infoSecurePort=0, ipcPort=40333, storageInfo=lv=-57;cid=testClusterID;nsid=1611104851;c=1693717223596), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:00:24,157 DEBUG [Listener at localhost/40333] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0 2023-09-03 05:00:24,160 INFO [Listener at localhost/40333] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/cluster_fce263f4-3716-36b0-1730-b29470dde46f/zookeeper_0, clientPort=57996, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/cluster_fce263f4-3716-36b0-1730-b29470dde46f/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/cluster_fce263f4-3716-36b0-1730-b29470dde46f/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-09-03 05:00:24,161 INFO [Listener at localhost/40333] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=57996 2023-09-03 05:00:24,161 INFO [Listener at localhost/40333] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:00:24,162 INFO [Listener at localhost/40333] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:00:24,178 INFO [Listener at localhost/40333] util.FSUtils(471): Created version file at hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849 with version=8 2023-09-03 05:00:24,178 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/hbase-staging 2023-09-03 05:00:24,180 INFO [Listener at localhost/40333] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-09-03 05:00:24,180 INFO [Listener at localhost/40333] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:00:24,180 INFO [Listener at localhost/40333] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-03 05:00:24,180 INFO [Listener at localhost/40333] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-03 05:00:24,180 INFO [Listener at localhost/40333] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:00:24,180 INFO [Listener at localhost/40333] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-03 05:00:24,180 INFO [Listener at localhost/40333] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-09-03 05:00:24,182 INFO [Listener at localhost/40333] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45873 2023-09-03 05:00:24,182 INFO [Listener at localhost/40333] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:00:24,183 INFO [Listener at localhost/40333] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:00:24,184 INFO [Listener at localhost/40333] zookeeper.RecoverableZooKeeper(93): Process identifier=master:45873 connecting to ZooKeeper ensemble=127.0.0.1:57996 2023-09-03 05:00:24,190 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:458730x0, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-03 05:00:24,191 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:45873-0x100702912850000 connected 2023-09-03 05:00:24,206 DEBUG [Listener at localhost/40333] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-03 05:00:24,206 DEBUG [Listener at localhost/40333] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:00:24,207 DEBUG [Listener at localhost/40333] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-03 05:00:24,213 DEBUG [Listener at localhost/40333] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45873 2023-09-03 05:00:24,213 DEBUG [Listener at localhost/40333] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45873 2023-09-03 05:00:24,216 DEBUG [Listener at localhost/40333] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45873 2023-09-03 05:00:24,217 DEBUG [Listener at localhost/40333] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45873 2023-09-03 05:00:24,218 DEBUG [Listener at localhost/40333] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45873 2023-09-03 05:00:24,218 INFO [Listener at localhost/40333] master.HMaster(444): hbase.rootdir=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849, hbase.cluster.distributed=false 2023-09-03 05:00:24,234 INFO [Listener at localhost/40333] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-09-03 05:00:24,234 INFO [Listener at localhost/40333] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:00:24,235 INFO [Listener at localhost/40333] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-03 05:00:24,235 INFO [Listener at localhost/40333] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-03 05:00:24,235 INFO [Listener at localhost/40333] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:00:24,235 INFO [Listener at localhost/40333] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-03 05:00:24,235 INFO [Listener at localhost/40333] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-09-03 05:00:24,237 INFO [Listener at localhost/40333] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42265 2023-09-03 05:00:24,238 INFO [Listener at localhost/40333] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-03 05:00:24,241 DEBUG [Listener at localhost/40333] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-09-03 05:00:24,242 INFO [Listener at localhost/40333] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:00:24,243 INFO [Listener at localhost/40333] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:00:24,244 INFO [Listener at localhost/40333] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42265 connecting to ZooKeeper ensemble=127.0.0.1:57996 2023-09-03 05:00:24,247 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:422650x0, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-03 05:00:24,248 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42265-0x100702912850001 connected 2023-09-03 05:00:24,249 DEBUG [Listener at localhost/40333] zookeeper.ZKUtil(164): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-03 05:00:24,249 DEBUG [Listener at localhost/40333] zookeeper.ZKUtil(164): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:00:24,250 DEBUG [Listener at localhost/40333] zookeeper.ZKUtil(164): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-03 05:00:24,253 DEBUG [Listener at localhost/40333] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42265 2023-09-03 05:00:24,253 DEBUG [Listener at localhost/40333] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42265 2023-09-03 05:00:24,253 DEBUG [Listener at localhost/40333] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42265 2023-09-03 05:00:24,254 DEBUG [Listener at localhost/40333] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42265 2023-09-03 05:00:24,254 DEBUG [Listener at localhost/40333] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42265 2023-09-03 05:00:24,255 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,45873,1693717224179 2023-09-03 05:00:24,257 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-03 05:00:24,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,45873,1693717224179 2023-09-03 05:00:24,259 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-03 05:00:24,259 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-03 05:00:24,259 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:24,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-03 05:00:24,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,45873,1693717224179 from backup master directory 2023-09-03 05:00:24,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-03 05:00:24,262 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,45873,1693717224179 2023-09-03 05:00:24,262 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-03 05:00:24,262 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-03 05:00:24,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,45873,1693717224179 2023-09-03 05:00:24,280 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/hbase.id with ID: 06b812f8-692f-4d2a-85e0-eb5b55eda7cb 2023-09-03 05:00:24,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:00:24,297 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:24,306 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x1a54c85b to 127.0.0.1:57996 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-03 05:00:24,310 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2378edf9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-03 05:00:24,310 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-03 05:00:24,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-09-03 05:00:24,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-03 05:00:24,313 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/MasterData/data/master/store-tmp 2023-09-03 05:00:24,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:24,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-03 05:00:24,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:00:24,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:00:24,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-03 05:00:24,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:00:24,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:00:24,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-03 05:00:24,324 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/MasterData/WALs/jenkins-hbase4.apache.org,45873,1693717224179 2023-09-03 05:00:24,327 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45873%2C1693717224179, suffix=, logDir=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/MasterData/WALs/jenkins-hbase4.apache.org,45873,1693717224179, archiveDir=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/MasterData/oldWALs, maxLogs=10 2023-09-03 05:00:24,343 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34017,DS-d7d9187f-2c3c-4690-b735-31baabf2e20c,DISK] 2023-09-03 05:00:24,343 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34525,DS-82878c9e-a256-484e-8cac-a79479bf91fd,DISK] 2023-09-03 05:00:24,346 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/MasterData/WALs/jenkins-hbase4.apache.org,45873,1693717224179/jenkins-hbase4.apache.org%2C45873%2C1693717224179.1693717224327 2023-09-03 05:00:24,346 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34017,DS-d7d9187f-2c3c-4690-b735-31baabf2e20c,DISK], DatanodeInfoWithStorage[127.0.0.1:34525,DS-82878c9e-a256-484e-8cac-a79479bf91fd,DISK]] 2023-09-03 05:00:24,346 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-09-03 05:00:24,346 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:24,347 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:00:24,347 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:00:24,349 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:00:24,351 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-09-03 05:00:24,351 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-09-03 05:00:24,352 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:24,353 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:00:24,354 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:00:24,357 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:00:24,360 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-03 05:00:24,361 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=691476, jitterRate=-0.12074367702007294}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-03 05:00:24,361 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-03 05:00:24,361 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-09-03 05:00:24,362 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-09-03 05:00:24,362 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-09-03 05:00:24,362 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-09-03 05:00:24,363 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-09-03 05:00:24,363 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 0 msec 2023-09-03 05:00:24,363 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-09-03 05:00:24,364 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-09-03 05:00:24,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-09-03 05:00:24,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-09-03 05:00:24,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-09-03 05:00:24,378 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-09-03 05:00:24,378 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-09-03 05:00:24,378 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-09-03 05:00:24,381 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:24,382 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-09-03 05:00:24,383 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-09-03 05:00:24,384 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-09-03 05:00:24,385 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-03 05:00:24,385 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-03 05:00:24,385 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:24,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,45873,1693717224179, sessionid=0x100702912850000, setting cluster-up flag (Was=false) 2023-09-03 05:00:24,390 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:24,394 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-09-03 05:00:24,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45873,1693717224179 2023-09-03 05:00:24,399 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:24,404 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-09-03 05:00:24,405 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45873,1693717224179 2023-09-03 05:00:24,406 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/.hbase-snapshot/.tmp 2023-09-03 05:00:24,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-09-03 05:00:24,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:00:24,409 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:00:24,409 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:00:24,409 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:00:24,409 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-09-03 05:00:24,409 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:24,409 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-03 05:00:24,409 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:24,410 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693717254410 2023-09-03 05:00:24,410 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-09-03 05:00:24,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-09-03 05:00:24,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-09-03 05:00:24,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-09-03 05:00:24,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-09-03 05:00:24,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-09-03 05:00:24,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:24,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-09-03 05:00:24,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-09-03 05:00:24,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-09-03 05:00:24,412 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-09-03 05:00:24,412 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-09-03 05:00:24,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-09-03 05:00:24,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-09-03 05:00:24,413 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693717224412,5,FailOnTimeoutGroup] 2023-09-03 05:00:24,413 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693717224413,5,FailOnTimeoutGroup] 2023-09-03 05:00:24,413 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:24,413 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-09-03 05:00:24,413 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:24,413 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:24,414 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-03 05:00:24,431 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-03 05:00:24,431 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-03 05:00:24,431 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849 2023-09-03 05:00:24,442 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:24,443 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-03 05:00:24,445 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/info 2023-09-03 05:00:24,446 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-03 05:00:24,447 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:24,447 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-03 05:00:24,448 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/rep_barrier 2023-09-03 05:00:24,449 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-03 05:00:24,449 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:24,450 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-03 05:00:24,451 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/table 2023-09-03 05:00:24,451 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-03 05:00:24,452 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:24,453 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740 2023-09-03 05:00:24,454 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740 2023-09-03 05:00:24,456 INFO [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(951): ClusterId : 06b812f8-692f-4d2a-85e0-eb5b55eda7cb 2023-09-03 05:00:24,457 DEBUG [RS:0;jenkins-hbase4:42265] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-09-03 05:00:24,458 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-03 05:00:24,459 DEBUG [RS:0;jenkins-hbase4:42265] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-09-03 05:00:24,459 DEBUG [RS:0;jenkins-hbase4:42265] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-09-03 05:00:24,460 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-03 05:00:24,462 DEBUG [RS:0;jenkins-hbase4:42265] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-09-03 05:00:24,463 DEBUG [RS:0;jenkins-hbase4:42265] zookeeper.ReadOnlyZKClient(139): Connect 0x4a1ffbb3 to 127.0.0.1:57996 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-03 05:00:24,463 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-03 05:00:24,464 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=795730, jitterRate=0.011823296546936035}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-03 05:00:24,465 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-03 05:00:24,466 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-03 05:00:24,466 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-03 05:00:24,466 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-03 05:00:24,466 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-03 05:00:24,466 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-03 05:00:24,467 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-03 05:00:24,467 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-03 05:00:24,468 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-09-03 05:00:24,468 DEBUG [RS:0;jenkins-hbase4:42265] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@44d81c80, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-03 05:00:24,468 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-09-03 05:00:24,468 DEBUG [RS:0;jenkins-hbase4:42265] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@40e790c3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-03 05:00:24,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-09-03 05:00:24,471 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-09-03 05:00:24,473 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-09-03 05:00:24,479 DEBUG [RS:0;jenkins-hbase4:42265] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:42265 2023-09-03 05:00:24,479 INFO [RS:0;jenkins-hbase4:42265] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-09-03 05:00:24,479 INFO [RS:0;jenkins-hbase4:42265] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-09-03 05:00:24,479 DEBUG [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(1022): About to register with Master. 2023-09-03 05:00:24,480 INFO [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45873,1693717224179 with isa=jenkins-hbase4.apache.org/172.31.14.131:42265, startcode=1693717224234 2023-09-03 05:00:24,480 DEBUG [RS:0;jenkins-hbase4:42265] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-09-03 05:00:24,483 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35413, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-09-03 05:00:24,484 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45873] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:24,485 DEBUG [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849 2023-09-03 05:00:24,485 DEBUG [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45657 2023-09-03 05:00:24,485 DEBUG [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-09-03 05:00:24,487 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-03 05:00:24,488 DEBUG [RS:0;jenkins-hbase4:42265] zookeeper.ZKUtil(162): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:24,488 WARN [RS:0;jenkins-hbase4:42265] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-03 05:00:24,488 INFO [RS:0;jenkins-hbase4:42265] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-03 05:00:24,488 DEBUG [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:24,489 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42265,1693717224234] 2023-09-03 05:00:24,492 DEBUG [RS:0;jenkins-hbase4:42265] zookeeper.ZKUtil(162): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:24,493 DEBUG [RS:0;jenkins-hbase4:42265] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-09-03 05:00:24,493 INFO [RS:0;jenkins-hbase4:42265] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-09-03 05:00:24,495 INFO [RS:0;jenkins-hbase4:42265] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-09-03 05:00:24,495 INFO [RS:0;jenkins-hbase4:42265] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-09-03 05:00:24,495 INFO [RS:0;jenkins-hbase4:42265] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:24,496 INFO [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-09-03 05:00:24,497 INFO [RS:0;jenkins-hbase4:42265] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:24,497 DEBUG [RS:0;jenkins-hbase4:42265] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:24,497 DEBUG [RS:0;jenkins-hbase4:42265] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:24,497 DEBUG [RS:0;jenkins-hbase4:42265] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:24,497 DEBUG [RS:0;jenkins-hbase4:42265] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:24,497 DEBUG [RS:0;jenkins-hbase4:42265] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:24,497 DEBUG [RS:0;jenkins-hbase4:42265] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-03 05:00:24,497 DEBUG [RS:0;jenkins-hbase4:42265] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:24,497 DEBUG [RS:0;jenkins-hbase4:42265] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:24,498 DEBUG [RS:0;jenkins-hbase4:42265] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:24,498 DEBUG [RS:0;jenkins-hbase4:42265] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:00:24,498 INFO [RS:0;jenkins-hbase4:42265] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:24,498 INFO [RS:0;jenkins-hbase4:42265] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:24,498 INFO [RS:0;jenkins-hbase4:42265] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:24,512 INFO [RS:0;jenkins-hbase4:42265] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-09-03 05:00:24,512 INFO [RS:0;jenkins-hbase4:42265] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42265,1693717224234-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:24,528 INFO [RS:0;jenkins-hbase4:42265] regionserver.Replication(203): jenkins-hbase4.apache.org,42265,1693717224234 started 2023-09-03 05:00:24,529 INFO [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,42265,1693717224234, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42265, sessionid=0x100702912850001 2023-09-03 05:00:24,529 DEBUG [RS:0;jenkins-hbase4:42265] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-09-03 05:00:24,529 DEBUG [RS:0;jenkins-hbase4:42265] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:24,529 DEBUG [RS:0;jenkins-hbase4:42265] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42265,1693717224234' 2023-09-03 05:00:24,529 DEBUG [RS:0;jenkins-hbase4:42265] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-03 05:00:24,529 DEBUG [RS:0;jenkins-hbase4:42265] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-03 05:00:24,530 DEBUG [RS:0;jenkins-hbase4:42265] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-09-03 05:00:24,530 DEBUG [RS:0;jenkins-hbase4:42265] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-09-03 05:00:24,530 DEBUG [RS:0;jenkins-hbase4:42265] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:24,530 DEBUG [RS:0;jenkins-hbase4:42265] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42265,1693717224234' 2023-09-03 05:00:24,530 DEBUG [RS:0;jenkins-hbase4:42265] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-09-03 05:00:24,530 DEBUG [RS:0;jenkins-hbase4:42265] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-09-03 05:00:24,531 DEBUG [RS:0;jenkins-hbase4:42265] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-09-03 05:00:24,531 INFO [RS:0;jenkins-hbase4:42265] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-09-03 05:00:24,531 INFO [RS:0;jenkins-hbase4:42265] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-09-03 05:00:24,624 DEBUG [jenkins-hbase4:45873] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-09-03 05:00:24,625 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42265,1693717224234, state=OPENING 2023-09-03 05:00:24,626 DEBUG [PEWorker-5] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-09-03 05:00:24,628 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:24,628 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-03 05:00:24,628 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42265,1693717224234}] 2023-09-03 05:00:24,634 INFO [RS:0;jenkins-hbase4:42265] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42265%2C1693717224234, suffix=, logDir=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234, archiveDir=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/oldWALs, maxLogs=32 2023-09-03 05:00:24,653 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34525,DS-82878c9e-a256-484e-8cac-a79479bf91fd,DISK] 2023-09-03 05:00:24,653 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34017,DS-d7d9187f-2c3c-4690-b735-31baabf2e20c,DISK] 2023-09-03 05:00:24,657 INFO [RS:0;jenkins-hbase4:42265] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234/jenkins-hbase4.apache.org%2C42265%2C1693717224234.1693717224636 2023-09-03 05:00:24,657 DEBUG [RS:0;jenkins-hbase4:42265] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34525,DS-82878c9e-a256-484e-8cac-a79479bf91fd,DISK], DatanodeInfoWithStorage[127.0.0.1:34017,DS-d7d9187f-2c3c-4690-b735-31baabf2e20c,DISK]] 2023-09-03 05:00:24,784 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:24,784 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-09-03 05:00:24,786 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41168, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-09-03 05:00:24,790 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-09-03 05:00:24,790 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-03 05:00:24,792 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42265%2C1693717224234.meta, suffix=.meta, logDir=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234, archiveDir=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/oldWALs, maxLogs=32 2023-09-03 05:00:24,808 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34017,DS-d7d9187f-2c3c-4690-b735-31baabf2e20c,DISK] 2023-09-03 05:00:24,808 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34525,DS-82878c9e-a256-484e-8cac-a79479bf91fd,DISK] 2023-09-03 05:00:24,810 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234/jenkins-hbase4.apache.org%2C42265%2C1693717224234.meta.1693717224793.meta 2023-09-03 05:00:24,811 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34017,DS-d7d9187f-2c3c-4690-b735-31baabf2e20c,DISK], DatanodeInfoWithStorage[127.0.0.1:34525,DS-82878c9e-a256-484e-8cac-a79479bf91fd,DISK]] 2023-09-03 05:00:24,811 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-09-03 05:00:24,812 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-09-03 05:00:24,812 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-09-03 05:00:24,813 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-09-03 05:00:24,813 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-09-03 05:00:24,813 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:24,813 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-09-03 05:00:24,813 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-09-03 05:00:24,815 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-03 05:00:24,816 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/info 2023-09-03 05:00:24,816 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/info 2023-09-03 05:00:24,816 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-03 05:00:24,817 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:24,817 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-03 05:00:24,818 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/rep_barrier 2023-09-03 05:00:24,818 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/rep_barrier 2023-09-03 05:00:24,819 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-03 05:00:24,819 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:24,819 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-03 05:00:24,820 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/table 2023-09-03 05:00:24,820 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/table 2023-09-03 05:00:24,821 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-03 05:00:24,822 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:24,823 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740 2023-09-03 05:00:24,825 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740 2023-09-03 05:00:24,827 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-03 05:00:24,829 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-03 05:00:24,830 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=853947, jitterRate=0.08585074543952942}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-03 05:00:24,830 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-03 05:00:24,831 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693717224784 2023-09-03 05:00:24,834 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-09-03 05:00:24,835 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-09-03 05:00:24,835 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42265,1693717224234, state=OPEN 2023-09-03 05:00:24,837 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-09-03 05:00:24,837 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-03 05:00:24,839 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-09-03 05:00:24,840 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42265,1693717224234 in 209 msec 2023-09-03 05:00:24,842 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-09-03 05:00:24,842 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 372 msec 2023-09-03 05:00:24,845 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 436 msec 2023-09-03 05:00:24,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693717224845, completionTime=-1 2023-09-03 05:00:24,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-09-03 05:00:24,845 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-09-03 05:00:24,848 DEBUG [hconnection-0x39aa3cfd-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-03 05:00:24,849 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41176, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-03 05:00:24,851 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-09-03 05:00:24,851 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693717284851 2023-09-03 05:00:24,851 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693717344851 2023-09-03 05:00:24,851 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-09-03 05:00:24,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45873,1693717224179-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:24,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45873,1693717224179-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:24,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45873,1693717224179-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:24,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:45873, period=300000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:24,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-09-03 05:00:24,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-09-03 05:00:24,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-03 05:00:24,860 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-09-03 05:00:24,860 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-09-03 05:00:24,862 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-09-03 05:00:24,863 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-03 05:00:24,865 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/.tmp/data/hbase/namespace/d49214d2a25145de199d28b106bafe32 2023-09-03 05:00:24,866 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/.tmp/data/hbase/namespace/d49214d2a25145de199d28b106bafe32 empty. 2023-09-03 05:00:24,866 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/.tmp/data/hbase/namespace/d49214d2a25145de199d28b106bafe32 2023-09-03 05:00:24,866 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-09-03 05:00:24,883 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-09-03 05:00:24,884 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => d49214d2a25145de199d28b106bafe32, NAME => 'hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/.tmp 2023-09-03 05:00:24,895 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:24,895 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing d49214d2a25145de199d28b106bafe32, disabling compactions & flushes 2023-09-03 05:00:24,895 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. 2023-09-03 05:00:24,895 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. 2023-09-03 05:00:24,895 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. after waiting 0 ms 2023-09-03 05:00:24,895 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. 2023-09-03 05:00:24,895 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. 2023-09-03 05:00:24,895 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for d49214d2a25145de199d28b106bafe32: 2023-09-03 05:00:24,898 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-09-03 05:00:24,900 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693717224900"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693717224900"}]},"ts":"1693717224900"} 2023-09-03 05:00:24,902 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-03 05:00:24,904 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-03 05:00:24,904 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693717224904"}]},"ts":"1693717224904"} 2023-09-03 05:00:24,905 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-09-03 05:00:24,913 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d49214d2a25145de199d28b106bafe32, ASSIGN}] 2023-09-03 05:00:24,915 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d49214d2a25145de199d28b106bafe32, ASSIGN 2023-09-03 05:00:24,916 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=d49214d2a25145de199d28b106bafe32, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42265,1693717224234; forceNewPlan=false, retain=false 2023-09-03 05:00:25,067 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=d49214d2a25145de199d28b106bafe32, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:25,067 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693717225067"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693717225067"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693717225067"}]},"ts":"1693717225067"} 2023-09-03 05:00:25,069 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure d49214d2a25145de199d28b106bafe32, server=jenkins-hbase4.apache.org,42265,1693717224234}] 2023-09-03 05:00:25,227 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. 2023-09-03 05:00:25,227 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d49214d2a25145de199d28b106bafe32, NAME => 'hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32.', STARTKEY => '', ENDKEY => ''} 2023-09-03 05:00:25,228 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace d49214d2a25145de199d28b106bafe32 2023-09-03 05:00:25,228 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:25,228 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d49214d2a25145de199d28b106bafe32 2023-09-03 05:00:25,228 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d49214d2a25145de199d28b106bafe32 2023-09-03 05:00:25,229 INFO [StoreOpener-d49214d2a25145de199d28b106bafe32-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region d49214d2a25145de199d28b106bafe32 2023-09-03 05:00:25,231 DEBUG [StoreOpener-d49214d2a25145de199d28b106bafe32-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/namespace/d49214d2a25145de199d28b106bafe32/info 2023-09-03 05:00:25,231 DEBUG [StoreOpener-d49214d2a25145de199d28b106bafe32-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/namespace/d49214d2a25145de199d28b106bafe32/info 2023-09-03 05:00:25,232 INFO [StoreOpener-d49214d2a25145de199d28b106bafe32-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d49214d2a25145de199d28b106bafe32 columnFamilyName info 2023-09-03 05:00:25,232 INFO [StoreOpener-d49214d2a25145de199d28b106bafe32-1] regionserver.HStore(310): Store=d49214d2a25145de199d28b106bafe32/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:25,233 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/namespace/d49214d2a25145de199d28b106bafe32 2023-09-03 05:00:25,234 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/namespace/d49214d2a25145de199d28b106bafe32 2023-09-03 05:00:25,237 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d49214d2a25145de199d28b106bafe32 2023-09-03 05:00:25,239 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/namespace/d49214d2a25145de199d28b106bafe32/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-03 05:00:25,240 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d49214d2a25145de199d28b106bafe32; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=847735, jitterRate=0.07795175909996033}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-03 05:00:25,240 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d49214d2a25145de199d28b106bafe32: 2023-09-03 05:00:25,241 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32., pid=6, masterSystemTime=1693717225222 2023-09-03 05:00:25,244 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. 2023-09-03 05:00:25,244 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. 2023-09-03 05:00:25,244 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=d49214d2a25145de199d28b106bafe32, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:25,245 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693717225244"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693717225244"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693717225244"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693717225244"}]},"ts":"1693717225244"} 2023-09-03 05:00:25,249 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-09-03 05:00:25,249 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure d49214d2a25145de199d28b106bafe32, server=jenkins-hbase4.apache.org,42265,1693717224234 in 178 msec 2023-09-03 05:00:25,252 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-09-03 05:00:25,253 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=d49214d2a25145de199d28b106bafe32, ASSIGN in 336 msec 2023-09-03 05:00:25,254 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-03 05:00:25,254 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693717225254"}]},"ts":"1693717225254"} 2023-09-03 05:00:25,256 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-09-03 05:00:25,258 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-09-03 05:00:25,260 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 399 msec 2023-09-03 05:00:25,261 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-09-03 05:00:25,263 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-09-03 05:00:25,263 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:25,268 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-09-03 05:00:25,276 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-03 05:00:25,280 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 11 msec 2023-09-03 05:00:25,290 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-09-03 05:00:25,297 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-03 05:00:25,302 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 11 msec 2023-09-03 05:00:25,314 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-09-03 05:00:25,317 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-09-03 05:00:25,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.055sec 2023-09-03 05:00:25,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-09-03 05:00:25,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-09-03 05:00:25,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-09-03 05:00:25,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45873,1693717224179-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-09-03 05:00:25,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45873,1693717224179-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-09-03 05:00:25,319 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-09-03 05:00:25,357 DEBUG [Listener at localhost/40333] zookeeper.ReadOnlyZKClient(139): Connect 0x55d1eb23 to 127.0.0.1:57996 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-03 05:00:25,361 DEBUG [Listener at localhost/40333] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f4bd615, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-03 05:00:25,363 DEBUG [hconnection-0x72048674-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-03 05:00:25,365 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35972, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-03 05:00:25,367 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,45873,1693717224179 2023-09-03 05:00:25,368 INFO [Listener at localhost/40333] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:00:25,372 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-09-03 05:00:25,372 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:00:25,373 INFO [Listener at localhost/40333] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-09-03 05:00:25,379 DEBUG [Listener at localhost/40333] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-09-03 05:00:25,382 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55902, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-09-03 05:00:25,384 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-09-03 05:00:25,384 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-09-03 05:00:25,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-03 05:00:25,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] procedure2.ProcedureExecutor(1028): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:25,391 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-09-03 05:00:25,391 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testCompactionRecordDoesntBlockRolling" procId is: 9 2023-09-03 05:00:25,392 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-03 05:00:25,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-03 05:00:25,395 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e 2023-09-03 05:00:25,396 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e empty. 2023-09-03 05:00:25,396 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e 2023-09-03 05:00:25,396 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testCompactionRecordDoesntBlockRolling regions 2023-09-03 05:00:25,431 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/.tabledesc/.tableinfo.0000000001 2023-09-03 05:00:25,433 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => b3c1671eba7bdf52eda9c2a56f0dc37e, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/.tmp 2023-09-03 05:00:25,463 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:25,463 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1604): Closing b3c1671eba7bdf52eda9c2a56f0dc37e, disabling compactions & flushes 2023-09-03 05:00:25,463 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:00:25,463 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:00:25,463 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. after waiting 0 ms 2023-09-03 05:00:25,463 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:00:25,463 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:00:25,463 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1558): Region close journal for b3c1671eba7bdf52eda9c2a56f0dc37e: 2023-09-03 05:00:25,474 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ADD_TO_META 2023-09-03 05:00:25,476 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1693717225476"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693717225476"}]},"ts":"1693717225476"} 2023-09-03 05:00:25,478 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-03 05:00:25,480 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-03 05:00:25,480 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693717225480"}]},"ts":"1693717225480"} 2023-09-03 05:00:25,482 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLING in hbase:meta 2023-09-03 05:00:25,488 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=b3c1671eba7bdf52eda9c2a56f0dc37e, ASSIGN}] 2023-09-03 05:00:25,490 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=b3c1671eba7bdf52eda9c2a56f0dc37e, ASSIGN 2023-09-03 05:00:25,491 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=b3c1671eba7bdf52eda9c2a56f0dc37e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42265,1693717224234; forceNewPlan=false, retain=false 2023-09-03 05:00:25,642 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=b3c1671eba7bdf52eda9c2a56f0dc37e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:25,643 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1693717225642"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693717225642"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693717225642"}]},"ts":"1693717225642"} 2023-09-03 05:00:25,645 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure b3c1671eba7bdf52eda9c2a56f0dc37e, server=jenkins-hbase4.apache.org,42265,1693717224234}] 2023-09-03 05:00:25,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:00:25,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b3c1671eba7bdf52eda9c2a56f0dc37e, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e.', STARTKEY => '', ENDKEY => ''} 2023-09-03 05:00:25,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testCompactionRecordDoesntBlockRolling b3c1671eba7bdf52eda9c2a56f0dc37e 2023-09-03 05:00:25,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:00:25,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b3c1671eba7bdf52eda9c2a56f0dc37e 2023-09-03 05:00:25,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b3c1671eba7bdf52eda9c2a56f0dc37e 2023-09-03 05:00:25,805 INFO [StoreOpener-b3c1671eba7bdf52eda9c2a56f0dc37e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region b3c1671eba7bdf52eda9c2a56f0dc37e 2023-09-03 05:00:25,807 DEBUG [StoreOpener-b3c1671eba7bdf52eda9c2a56f0dc37e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info 2023-09-03 05:00:25,807 DEBUG [StoreOpener-b3c1671eba7bdf52eda9c2a56f0dc37e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info 2023-09-03 05:00:25,807 INFO [StoreOpener-b3c1671eba7bdf52eda9c2a56f0dc37e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b3c1671eba7bdf52eda9c2a56f0dc37e columnFamilyName info 2023-09-03 05:00:25,808 INFO [StoreOpener-b3c1671eba7bdf52eda9c2a56f0dc37e-1] regionserver.HStore(310): Store=b3c1671eba7bdf52eda9c2a56f0dc37e/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:00:25,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e 2023-09-03 05:00:25,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e 2023-09-03 05:00:25,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b3c1671eba7bdf52eda9c2a56f0dc37e 2023-09-03 05:00:25,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-03 05:00:25,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b3c1671eba7bdf52eda9c2a56f0dc37e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=845116, jitterRate=0.07462070882320404}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-03 05:00:25,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b3c1671eba7bdf52eda9c2a56f0dc37e: 2023-09-03 05:00:25,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e., pid=11, masterSystemTime=1693717225798 2023-09-03 05:00:25,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:00:25,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:00:25,820 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=b3c1671eba7bdf52eda9c2a56f0dc37e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:25,820 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1693717225819"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693717225819"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693717225819"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693717225819"}]},"ts":"1693717225819"} 2023-09-03 05:00:25,824 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=11, resume processing ppid=10 2023-09-03 05:00:25,824 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure b3c1671eba7bdf52eda9c2a56f0dc37e, server=jenkins-hbase4.apache.org,42265,1693717224234 in 177 msec 2023-09-03 05:00:25,827 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=10, resume processing ppid=9 2023-09-03 05:00:25,827 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=b3c1671eba7bdf52eda9c2a56f0dc37e, ASSIGN in 336 msec 2023-09-03 05:00:25,828 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-03 05:00:25,828 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693717225828"}]},"ts":"1693717225828"} 2023-09-03 05:00:25,830 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLED in hbase:meta 2023-09-03 05:00:25,832 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_POST_OPERATION 2023-09-03 05:00:25,834 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling in 447 msec 2023-09-03 05:00:25,851 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-03 05:00:29,824 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-03 05:00:29,824 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-09-03 05:00:30,493 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-09-03 05:00:30,494 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-09-03 05:00:30,494 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-03 05:00:31,307 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-03 05:00:35,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-03 05:00:35,395 INFO [Listener at localhost/40333] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testCompactionRecordDoesntBlockRolling, procId: 9 completed 2023-09-03 05:00:35,398 DEBUG [Listener at localhost/40333] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:35,398 DEBUG [Listener at localhost/40333] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:00:35,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-09-03 05:00:35,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] procedure.ProcedureCoordinator(165): Submitting procedure hbase:namespace 2023-09-03 05:00:35,418 INFO [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'hbase:namespace' 2023-09-03 05:00:35,418 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-03 05:00:35,419 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'hbase:namespace' starting 'acquire' 2023-09-03 05:00:35,419 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'hbase:namespace', kicking off acquire phase on members. 2023-09-03 05:00:35,419 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-09-03 05:00:35,419 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-09-03 05:00:35,421 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-03 05:00:35,421 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-03 05:00:35,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-03 05:00:35,422 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,422 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-09-03 05:00:35,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/hbase:namespace 2023-09-03 05:00:35,422 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-09-03 05:00:35,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-09-03 05:00:35,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-09-03 05:00:35,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for hbase:namespace 2023-09-03 05:00:35,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:hbase:namespace 2023-09-03 05:00:35,425 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'hbase:namespace' with timeout 60000ms 2023-09-03 05:00:35,425 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-03 05:00:35,426 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'hbase:namespace' starting 'acquire' stage 2023-09-03 05:00:35,426 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-09-03 05:00:35,426 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. 2023-09-03 05:00:35,426 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-09-03 05:00:35,426 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. started... 2023-09-03 05:00:35,427 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d49214d2a25145de199d28b106bafe32 1/1 column families, dataSize=78 B heapSize=488 B 2023-09-03 05:00:35,439 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/namespace/d49214d2a25145de199d28b106bafe32/.tmp/info/3f935251e55e4d83a1f319cae9af7bb6 2023-09-03 05:00:35,447 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/namespace/d49214d2a25145de199d28b106bafe32/.tmp/info/3f935251e55e4d83a1f319cae9af7bb6 as hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/namespace/d49214d2a25145de199d28b106bafe32/info/3f935251e55e4d83a1f319cae9af7bb6 2023-09-03 05:00:35,455 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/namespace/d49214d2a25145de199d28b106bafe32/info/3f935251e55e4d83a1f319cae9af7bb6, entries=2, sequenceid=6, filesize=4.8 K 2023-09-03 05:00:35,456 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for d49214d2a25145de199d28b106bafe32 in 29ms, sequenceid=6, compaction requested=false 2023-09-03 05:00:35,457 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d49214d2a25145de199d28b106bafe32: 2023-09-03 05:00:35,457 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. 2023-09-03 05:00:35,457 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-09-03 05:00:35,457 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-09-03 05:00:35,457 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,457 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'hbase:namespace' locally acquired 2023-09-03 05:00:35,457 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42265,1693717224234' joining acquired barrier for procedure (hbase:namespace) in zk 2023-09-03 05:00:35,459 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,459 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/hbase:namespace 2023-09-03 05:00:35,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-03 05:00:35,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-03 05:00:35,460 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace 2023-09-03 05:00:35,460 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'hbase:namespace' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-09-03 05:00:35,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-03 05:00:35,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-03 05:00:35,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-03 05:00:35,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-03 05:00:35,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42265,1693717224234' joining acquired barrier for procedure 'hbase:namespace' on coordinator 2023-09-03 05:00:35,461 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'hbase:namespace' starting 'in-barrier' execution. 2023-09-03 05:00:35,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b332643[Count = 0] remaining members to acquire global barrier 2023-09-03 05:00:35,461 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/hbase:namespace 2023-09-03 05:00:35,464 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-09-03 05:00:35,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/hbase:namespace 2023-09-03 05:00:35,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/hbase:namespace 2023-09-03 05:00:35,464 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'hbase:namespace' received 'reached' from coordinator. 2023-09-03 05:00:35,464 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,464 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'hbase:namespace' locally completed 2023-09-03 05:00:35,464 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-09-03 05:00:35,464 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'hbase:namespace' completed for member 'jenkins-hbase4.apache.org,42265,1693717224234' in zk 2023-09-03 05:00:35,466 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,466 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'hbase:namespace' has notified controller of completion 2023-09-03 05:00:35,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,466 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-03 05:00:35,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-03 05:00:35,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-03 05:00:35,466 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'hbase:namespace' completed. 2023-09-03 05:00:35,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-03 05:00:35,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-03 05:00:35,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-03 05:00:35,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-03 05:00:35,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-03 05:00:35,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'hbase:namespace' member 'jenkins-hbase4.apache.org,42265,1693717224234': 2023-09-03 05:00:35,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42265,1693717224234' released barrier for procedure'hbase:namespace', counting down latch. Waiting for 0 more 2023-09-03 05:00:35,469 INFO [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'hbase:namespace' execution completed 2023-09-03 05:00:35,470 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-09-03 05:00:35,470 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-09-03 05:00:35,470 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:hbase:namespace 2023-09-03 05:00:35,470 INFO [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure hbase:namespaceincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-09-03 05:00:35,473 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-09-03 05:00:35,473 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-09-03 05:00:35,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/hbase:namespace 2023-09-03 05:00:35,473 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-03 05:00:35,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-09-03 05:00:35,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/hbase:namespace 2023-09-03 05:00:35,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-03 05:00:35,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-03 05:00:35,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-03 05:00:35,474 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-03 05:00:35,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-03 05:00:35,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-09-03 05:00:35,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-03 05:00:35,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-03 05:00:35,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-03 05:00:35,475 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-03 05:00:35,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-03 05:00:35,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,481 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-03 05:00:35,481 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-03 05:00:35,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-03 05:00:35,481 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-03 05:00:35,481 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-03 05:00:35,481 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'hbase:namespace' 2023-09-03 05:00:35,481 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-09-03 05:00:35,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-09-03 05:00:35,482 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-09-03 05:00:35,482 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:35,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-03 05:00:35,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-03 05:00:35,482 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-09-03 05:00:35,483 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-09-03 05:00:35,483 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-09-03 05:00:35,484 DEBUG [Listener at localhost/40333] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : hbase:namespace'' to complete. (max 20000 ms per retry) 2023-09-03 05:00:35,484 DEBUG [Listener at localhost/40333] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-09-03 05:00:45,484 DEBUG [Listener at localhost/40333] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-09-03 05:00:45,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-09-03 05:00:45,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-09-03 05:00:45,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,503 INFO [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-03 05:00:45,503 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-03 05:00:45,503 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-09-03 05:00:45,503 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-09-03 05:00:45,504 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,504 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,506 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-03 05:00:45,506 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-03 05:00:45,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-03 05:00:45,506 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,506 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-09-03 05:00:45,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,507 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-09-03 05:00:45,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,507 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,507 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,507 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-09-03 05:00:45,507 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-03 05:00:45,508 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-09-03 05:00:45,508 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-09-03 05:00:45,508 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-09-03 05:00:45,508 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:00:45,508 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. started... 2023-09-03 05:00:45,508 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b3c1671eba7bdf52eda9c2a56f0dc37e 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-03 05:00:45,523 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=5 (bloomFilter=true), to=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/.tmp/info/b4201e1b9de74d79b35bea026fa6b51d 2023-09-03 05:00:45,534 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/.tmp/info/b4201e1b9de74d79b35bea026fa6b51d as hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/b4201e1b9de74d79b35bea026fa6b51d 2023-09-03 05:00:45,543 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/b4201e1b9de74d79b35bea026fa6b51d, entries=1, sequenceid=5, filesize=5.8 K 2023-09-03 05:00:45,545 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for b3c1671eba7bdf52eda9c2a56f0dc37e in 37ms, sequenceid=5, compaction requested=false 2023-09-03 05:00:45,545 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b3c1671eba7bdf52eda9c2a56f0dc37e: 2023-09-03 05:00:45,545 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:00:45,545 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-09-03 05:00:45,546 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-09-03 05:00:45,546 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,546 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-09-03 05:00:45,546 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42265,1693717224234' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-09-03 05:00:45,548 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,548 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-03 05:00:45,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-03 05:00:45,549 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,549 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-09-03 05:00:45,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-03 05:00:45,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-03 05:00:45,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-03 05:00:45,550 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42265,1693717224234' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-09-03 05:00:45,550 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f0cb1fd[Count = 0] remaining members to acquire global barrier 2023-09-03 05:00:45,550 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-09-03 05:00:45,551 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,552 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,552 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-09-03 05:00:45,552 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-09-03 05:00:45,552 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,42265,1693717224234' in zk 2023-09-03 05:00:45,552 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,552 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-09-03 05:00:45,553 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-09-03 05:00:45,553 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,553 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-03 05:00:45,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-03 05:00:45,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-03 05:00:45,553 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-09-03 05:00:45,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-03 05:00:45,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-03 05:00:45,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-03 05:00:45,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,42265,1693717224234': 2023-09-03 05:00:45,557 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42265,1693717224234' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-09-03 05:00:45,557 INFO [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-09-03 05:00:45,557 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-09-03 05:00:45,557 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-09-03 05:00:45,557 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,557 INFO [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-09-03 05:00:45,562 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,562 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-03 05:00:45,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-03 05:00:45,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,562 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-03 05:00:45,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-03 05:00:45,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-03 05:00:45,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-03 05:00:45,562 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-03 05:00:45,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,564 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-03 05:00:45,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,567 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,567 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-03 05:00:45,567 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-03 05:00:45,567 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-03 05:00:45,567 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-03 05:00:45,567 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-09-03 05:00:45,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-03 05:00:45,567 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-03 05:00:45,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-09-03 05:00:45,567 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,568 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:45,568 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,568 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,568 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:45,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-03 05:00:45,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-03 05:00:45,568 DEBUG [Listener at localhost/40333] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-09-03 05:00:45,568 DEBUG [Listener at localhost/40333] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-09-03 05:00:55,568 DEBUG [Listener at localhost/40333] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-09-03 05:00:55,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-09-03 05:00:55,576 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-09-03 05:00:55,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-09-03 05:00:55,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,578 INFO [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-03 05:00:55,578 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-03 05:00:55,579 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-09-03 05:00:55,579 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-09-03 05:00:55,580 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,580 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,582 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-03 05:00:55,582 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-03 05:00:55,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-03 05:00:55,583 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,583 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-09-03 05:00:55,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,583 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-09-03 05:00:55,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,584 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,584 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-09-03 05:00:55,584 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,584 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-09-03 05:00:55,584 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-03 05:00:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-09-03 05:00:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-09-03 05:00:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-09-03 05:00:55,585 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:00:55,585 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. started... 2023-09-03 05:00:55,585 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b3c1671eba7bdf52eda9c2a56f0dc37e 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-03 05:00:55,600 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=9 (bloomFilter=true), to=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/.tmp/info/e1d62ce75de946c0a0e890350332abf9 2023-09-03 05:00:55,607 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/.tmp/info/e1d62ce75de946c0a0e890350332abf9 as hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/e1d62ce75de946c0a0e890350332abf9 2023-09-03 05:00:55,616 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/e1d62ce75de946c0a0e890350332abf9, entries=1, sequenceid=9, filesize=5.8 K 2023-09-03 05:00:55,617 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for b3c1671eba7bdf52eda9c2a56f0dc37e in 32ms, sequenceid=9, compaction requested=false 2023-09-03 05:00:55,617 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b3c1671eba7bdf52eda9c2a56f0dc37e: 2023-09-03 05:00:55,617 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:00:55,617 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-09-03 05:00:55,618 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-09-03 05:00:55,618 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,618 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-09-03 05:00:55,618 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42265,1693717224234' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-09-03 05:00:55,619 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,619 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-03 05:00:55,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-03 05:00:55,620 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,620 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-09-03 05:00:55,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-03 05:00:55,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-03 05:00:55,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-03 05:00:55,622 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42265,1693717224234' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-09-03 05:00:55,622 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a0e2c78[Count = 0] remaining members to acquire global barrier 2023-09-03 05:00:55,622 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-09-03 05:00:55,622 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,623 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,623 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-09-03 05:00:55,623 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-09-03 05:00:55,624 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,624 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-09-03 05:00:55,624 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,42265,1693717224234' in zk 2023-09-03 05:00:55,625 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-09-03 05:00:55,625 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-03 05:00:55,625 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,625 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-09-03 05:00:55,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-03 05:00:55,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-03 05:00:55,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-03 05:00:55,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-03 05:00:55,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-03 05:00:55,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,42265,1693717224234': 2023-09-03 05:00:55,629 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42265,1693717224234' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-09-03 05:00:55,629 INFO [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-09-03 05:00:55,629 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-09-03 05:00:55,629 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-09-03 05:00:55,629 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,629 INFO [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-09-03 05:00:55,632 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,632 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,632 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-03 05:00:55,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-03 05:00:55,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-03 05:00:55,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-03 05:00:55,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-03 05:00:55,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-03 05:00:55,633 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-03 05:00:55,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,634 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-03 05:00:55,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,638 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,638 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,638 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,638 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:00:55,638 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,638 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-03 05:00:55,638 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-03 05:00:55,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-03 05:00:55,638 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,638 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-03 05:00:55,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-03 05:00:55,638 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:00:55,638 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-03 05:00:55,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-09-03 05:00:55,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-03 05:00:55,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-03 05:00:55,639 DEBUG [Listener at localhost/40333] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-09-03 05:00:55,640 DEBUG [Listener at localhost/40333] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-09-03 05:01:05,640 DEBUG [Listener at localhost/40333] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-09-03 05:01:05,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-09-03 05:01:05,660 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34525,DS-82878c9e-a256-484e-8cac-a79479bf91fd,DISK] 2023-09-03 05:01:05,660 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34017,DS-d7d9187f-2c3c-4690-b735-31baabf2e20c,DISK] 2023-09-03 05:01:05,664 INFO [Listener at localhost/40333] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234/jenkins-hbase4.apache.org%2C42265%2C1693717224234.1693717224636 with entries=13, filesize=6.36 KB; new WAL /user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234/jenkins-hbase4.apache.org%2C42265%2C1693717224234.1693717265644 2023-09-03 05:01:05,664 DEBUG [Listener at localhost/40333] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34525,DS-82878c9e-a256-484e-8cac-a79479bf91fd,DISK], DatanodeInfoWithStorage[127.0.0.1:34017,DS-d7d9187f-2c3c-4690-b735-31baabf2e20c,DISK]] 2023-09-03 05:01:05,665 DEBUG [Listener at localhost/40333] wal.AbstractFSWAL(716): hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234/jenkins-hbase4.apache.org%2C42265%2C1693717224234.1693717224636 is not closed yet, will try archiving it next time 2023-09-03 05:01:05,670 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-09-03 05:01:05,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-09-03 05:01:05,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,672 INFO [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-03 05:01:05,672 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-03 05:01:05,673 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-09-03 05:01:05,673 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-09-03 05:01:05,673 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,673 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,675 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,675 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-03 05:01:05,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-03 05:01:05,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-03 05:01:05,675 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,675 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-09-03 05:01:05,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,675 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-09-03 05:01:05,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,676 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,676 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-09-03 05:01:05,676 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,676 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-09-03 05:01:05,676 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-03 05:01:05,677 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-09-03 05:01:05,677 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-09-03 05:01:05,677 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-09-03 05:01:05,677 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:01:05,677 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. started... 2023-09-03 05:01:05,677 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b3c1671eba7bdf52eda9c2a56f0dc37e 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-03 05:01:05,688 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=13 (bloomFilter=true), to=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/.tmp/info/184f2a50528844529e6519a9d32c1012 2023-09-03 05:01:05,695 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/.tmp/info/184f2a50528844529e6519a9d32c1012 as hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/184f2a50528844529e6519a9d32c1012 2023-09-03 05:01:05,703 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/184f2a50528844529e6519a9d32c1012, entries=1, sequenceid=13, filesize=5.8 K 2023-09-03 05:01:05,704 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for b3c1671eba7bdf52eda9c2a56f0dc37e in 27ms, sequenceid=13, compaction requested=true 2023-09-03 05:01:05,704 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b3c1671eba7bdf52eda9c2a56f0dc37e: 2023-09-03 05:01:05,704 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:01:05,704 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-09-03 05:01:05,705 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-09-03 05:01:05,705 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,705 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-09-03 05:01:05,705 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42265,1693717224234' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-09-03 05:01:05,707 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,707 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-03 05:01:05,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-03 05:01:05,707 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,707 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-09-03 05:01:05,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-03 05:01:05,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-03 05:01:05,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-03 05:01:05,708 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42265,1693717224234' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-09-03 05:01:05,709 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@699da506[Count = 0] remaining members to acquire global barrier 2023-09-03 05:01:05,709 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-09-03 05:01:05,709 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,711 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,711 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-09-03 05:01:05,711 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-09-03 05:01:05,711 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,42265,1693717224234' in zk 2023-09-03 05:01:05,711 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,711 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-09-03 05:01:05,713 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,713 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-09-03 05:01:05,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-03 05:01:05,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-03 05:01:05,713 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-03 05:01:05,713 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-09-03 05:01:05,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-03 05:01:05,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-03 05:01:05,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-03 05:01:05,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,42265,1693717224234': 2023-09-03 05:01:05,716 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42265,1693717224234' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-09-03 05:01:05,716 INFO [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-09-03 05:01:05,716 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-09-03 05:01:05,716 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-09-03 05:01:05,716 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,716 INFO [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-09-03 05:01:05,717 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,718 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,718 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-03 05:01:05,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-03 05:01:05,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-03 05:01:05,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-03 05:01:05,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-03 05:01:05,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-03 05:01:05,718 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-03 05:01:05,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,720 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-03 05:01:05,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,722 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,722 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-03 05:01:05,722 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-03 05:01:05,722 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,722 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-03 05:01:05,722 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-09-03 05:01:05,722 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-03 05:01:05,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-03 05:01:05,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-09-03 05:01:05,722 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:05,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-03 05:01:05,723 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,723 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,723 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:05,723 DEBUG [Listener at localhost/40333] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-09-03 05:01:05,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-03 05:01:05,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-03 05:01:05,723 DEBUG [Listener at localhost/40333] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-09-03 05:01:15,724 DEBUG [Listener at localhost/40333] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-09-03 05:01:15,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-09-03 05:01:15,727 DEBUG [Listener at localhost/40333] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-03 05:01:15,732 DEBUG [Listener at localhost/40333] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 17694 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-03 05:01:15,734 DEBUG [Listener at localhost/40333] regionserver.HStore(1912): b3c1671eba7bdf52eda9c2a56f0dc37e/info is initiating minor compaction (all files) 2023-09-03 05:01:15,735 INFO [Listener at localhost/40333] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-09-03 05:01:15,735 INFO [Listener at localhost/40333] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:15,735 INFO [Listener at localhost/40333] regionserver.HRegion(2259): Starting compaction of b3c1671eba7bdf52eda9c2a56f0dc37e/info in TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:01:15,735 INFO [Listener at localhost/40333] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/b4201e1b9de74d79b35bea026fa6b51d, hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/e1d62ce75de946c0a0e890350332abf9, hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/184f2a50528844529e6519a9d32c1012] into tmpdir=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/.tmp, totalSize=17.3 K 2023-09-03 05:01:15,736 DEBUG [Listener at localhost/40333] compactions.Compactor(207): Compacting b4201e1b9de74d79b35bea026fa6b51d, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1693717245495 2023-09-03 05:01:15,737 DEBUG [Listener at localhost/40333] compactions.Compactor(207): Compacting e1d62ce75de946c0a0e890350332abf9, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1693717255571 2023-09-03 05:01:15,737 DEBUG [Listener at localhost/40333] compactions.Compactor(207): Compacting 184f2a50528844529e6519a9d32c1012, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1693717265642 2023-09-03 05:01:15,759 INFO [Listener at localhost/40333] throttle.PressureAwareThroughputController(145): b3c1671eba7bdf52eda9c2a56f0dc37e#info#compaction#9 average throughput is 3.01 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-03 05:01:15,779 DEBUG [Listener at localhost/40333] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/.tmp/info/4aa64b231a6a45718a909cd8536108b9 as hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/4aa64b231a6a45718a909cd8536108b9 2023-09-03 05:01:15,795 INFO [Listener at localhost/40333] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in b3c1671eba7bdf52eda9c2a56f0dc37e/info of b3c1671eba7bdf52eda9c2a56f0dc37e into 4aa64b231a6a45718a909cd8536108b9(size=7.9 K), total size for store is 7.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-03 05:01:15,795 DEBUG [Listener at localhost/40333] regionserver.HRegion(2289): Compaction status journal for b3c1671eba7bdf52eda9c2a56f0dc37e: 2023-09-03 05:01:15,815 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34017,DS-d7d9187f-2c3c-4690-b735-31baabf2e20c,DISK] 2023-09-03 05:01:15,816 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34525,DS-82878c9e-a256-484e-8cac-a79479bf91fd,DISK] 2023-09-03 05:01:15,818 INFO [Listener at localhost/40333] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234/jenkins-hbase4.apache.org%2C42265%2C1693717224234.1693717265644 with entries=4, filesize=2.43 KB; new WAL /user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234/jenkins-hbase4.apache.org%2C42265%2C1693717224234.1693717275798 2023-09-03 05:01:15,818 DEBUG [Listener at localhost/40333] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34017,DS-d7d9187f-2c3c-4690-b735-31baabf2e20c,DISK], DatanodeInfoWithStorage[127.0.0.1:34525,DS-82878c9e-a256-484e-8cac-a79479bf91fd,DISK]] 2023-09-03 05:01:15,819 DEBUG [Listener at localhost/40333] wal.AbstractFSWAL(716): hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234/jenkins-hbase4.apache.org%2C42265%2C1693717224234.1693717265644 is not closed yet, will try archiving it next time 2023-09-03 05:01:15,820 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234/jenkins-hbase4.apache.org%2C42265%2C1693717224234.1693717224636 to hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/oldWALs/jenkins-hbase4.apache.org%2C42265%2C1693717224234.1693717224636 2023-09-03 05:01:15,826 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-09-03 05:01:15,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-09-03 05:01:15,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,829 INFO [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-03 05:01:15,829 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-03 05:01:15,829 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-09-03 05:01:15,829 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-09-03 05:01:15,830 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,830 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,835 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,835 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-03 05:01:15,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-03 05:01:15,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-03 05:01:15,835 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,835 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-09-03 05:01:15,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,836 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-09-03 05:01:15,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,836 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,837 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-09-03 05:01:15,837 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,837 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-09-03 05:01:15,837 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-03 05:01:15,837 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-09-03 05:01:15,837 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-09-03 05:01:15,837 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-09-03 05:01:15,837 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:01:15,838 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. started... 2023-09-03 05:01:15,838 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b3c1671eba7bdf52eda9c2a56f0dc37e 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-03 05:01:15,874 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=18 (bloomFilter=true), to=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/.tmp/info/dad617025ebf4609b7b23e475329f6a6 2023-09-03 05:01:15,881 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/.tmp/info/dad617025ebf4609b7b23e475329f6a6 as hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/dad617025ebf4609b7b23e475329f6a6 2023-09-03 05:01:15,888 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/dad617025ebf4609b7b23e475329f6a6, entries=1, sequenceid=18, filesize=5.8 K 2023-09-03 05:01:15,889 INFO [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for b3c1671eba7bdf52eda9c2a56f0dc37e in 51ms, sequenceid=18, compaction requested=false 2023-09-03 05:01:15,889 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b3c1671eba7bdf52eda9c2a56f0dc37e: 2023-09-03 05:01:15,889 DEBUG [rs(jenkins-hbase4.apache.org,42265,1693717224234)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:01:15,889 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-09-03 05:01:15,890 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-09-03 05:01:15,890 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,890 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-09-03 05:01:15,890 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42265,1693717224234' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-09-03 05:01:15,892 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,892 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-03 05:01:15,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-03 05:01:15,892 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,892 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-09-03 05:01:15,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-03 05:01:15,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-03 05:01:15,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-03 05:01:15,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42265,1693717224234' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-09-03 05:01:15,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b9e9748[Count = 0] remaining members to acquire global barrier 2023-09-03 05:01:15,894 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-09-03 05:01:15,894 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,895 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,895 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-09-03 05:01:15,895 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-09-03 05:01:15,895 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,42265,1693717224234' in zk 2023-09-03 05:01:15,895 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,895 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-09-03 05:01:15,897 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,897 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-09-03 05:01:15,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-03 05:01:15,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-03 05:01:15,897 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-03 05:01:15,897 DEBUG [member: 'jenkins-hbase4.apache.org,42265,1693717224234' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-09-03 05:01:15,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-03 05:01:15,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-03 05:01:15,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-03 05:01:15,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,42265,1693717224234': 2023-09-03 05:01:15,900 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42265,1693717224234' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-09-03 05:01:15,900 INFO [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-09-03 05:01:15,900 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-09-03 05:01:15,900 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-09-03 05:01:15,900 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,900 INFO [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-09-03 05:01:15,903 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,903 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-03 05:01:15,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-03 05:01:15,904 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-03 05:01:15,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,904 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-03 05:01:15,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-03 05:01:15,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-03 05:01:15,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-03 05:01:15,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,906 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-03 05:01:15,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,908 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,908 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-03 05:01:15,908 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,909 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-03 05:01:15,909 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,909 DEBUG [(jenkins-hbase4.apache.org,45873,1693717224179)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-03 05:01:15,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-09-03 05:01:15,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-03 05:01:15,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-03 05:01:15,909 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-03 05:01:15,909 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:15,909 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-09-03 05:01:15,910 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,910 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,910 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-03 05:01:15,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-03 05:01:15,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-03 05:01:15,910 DEBUG [Listener at localhost/40333] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-09-03 05:01:15,910 DEBUG [Listener at localhost/40333] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-09-03 05:01:25,910 DEBUG [Listener at localhost/40333] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-09-03 05:01:25,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45873] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-09-03 05:01:25,930 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34525,DS-82878c9e-a256-484e-8cac-a79479bf91fd,DISK] 2023-09-03 05:01:25,931 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34017,DS-d7d9187f-2c3c-4690-b735-31baabf2e20c,DISK] 2023-09-03 05:01:25,933 INFO [Listener at localhost/40333] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234/jenkins-hbase4.apache.org%2C42265%2C1693717224234.1693717275798 with entries=3, filesize=1.94 KB; new WAL /user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234/jenkins-hbase4.apache.org%2C42265%2C1693717224234.1693717285914 2023-09-03 05:01:25,933 DEBUG [Listener at localhost/40333] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34525,DS-82878c9e-a256-484e-8cac-a79479bf91fd,DISK], DatanodeInfoWithStorage[127.0.0.1:34017,DS-d7d9187f-2c3c-4690-b735-31baabf2e20c,DISK]] 2023-09-03 05:01:25,933 DEBUG [Listener at localhost/40333] wal.AbstractFSWAL(716): hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234/jenkins-hbase4.apache.org%2C42265%2C1693717224234.1693717275798 is not closed yet, will try archiving it next time 2023-09-03 05:01:25,934 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-09-03 05:01:25,934 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/WALs/jenkins-hbase4.apache.org,42265,1693717224234/jenkins-hbase4.apache.org%2C42265%2C1693717224234.1693717265644 to hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/oldWALs/jenkins-hbase4.apache.org%2C42265%2C1693717224234.1693717265644 2023-09-03 05:01:25,934 INFO [Listener at localhost/40333] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-09-03 05:01:25,934 DEBUG [Listener at localhost/40333] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x55d1eb23 to 127.0.0.1:57996 2023-09-03 05:01:25,934 DEBUG [Listener at localhost/40333] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:01:25,935 DEBUG [Listener at localhost/40333] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-09-03 05:01:25,935 DEBUG [Listener at localhost/40333] util.JVMClusterUtil(257): Found active master hash=790090134, stopped=false 2023-09-03 05:01:25,935 INFO [Listener at localhost/40333] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,45873,1693717224179 2023-09-03 05:01:25,937 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-03 05:01:25,937 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-03 05:01:25,937 INFO [Listener at localhost/40333] procedure2.ProcedureExecutor(628): Stopping 2023-09-03 05:01:25,937 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:01:25,938 DEBUG [Listener at localhost/40333] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1a54c85b to 127.0.0.1:57996 2023-09-03 05:01:25,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:01:25,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:01:25,938 DEBUG [Listener at localhost/40333] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:01:25,938 INFO [Listener at localhost/40333] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42265,1693717224234' ***** 2023-09-03 05:01:25,938 INFO [Listener at localhost/40333] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-09-03 05:01:25,939 INFO [RS:0;jenkins-hbase4:42265] regionserver.HeapMemoryManager(220): Stopping 2023-09-03 05:01:25,939 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-09-03 05:01:25,939 INFO [RS:0;jenkins-hbase4:42265] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-09-03 05:01:25,939 INFO [RS:0;jenkins-hbase4:42265] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-09-03 05:01:25,939 INFO [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(3305): Received CLOSE for d49214d2a25145de199d28b106bafe32 2023-09-03 05:01:25,940 INFO [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(3305): Received CLOSE for b3c1671eba7bdf52eda9c2a56f0dc37e 2023-09-03 05:01:25,940 INFO [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:25,940 DEBUG [RS:0;jenkins-hbase4:42265] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4a1ffbb3 to 127.0.0.1:57996 2023-09-03 05:01:25,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d49214d2a25145de199d28b106bafe32, disabling compactions & flushes 2023-09-03 05:01:25,940 DEBUG [RS:0;jenkins-hbase4:42265] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:01:25,940 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. 2023-09-03 05:01:25,940 INFO [RS:0;jenkins-hbase4:42265] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-09-03 05:01:25,940 INFO [RS:0;jenkins-hbase4:42265] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-09-03 05:01:25,940 INFO [RS:0;jenkins-hbase4:42265] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-09-03 05:01:25,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. 2023-09-03 05:01:25,940 INFO [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-09-03 05:01:25,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. after waiting 0 ms 2023-09-03 05:01:25,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. 2023-09-03 05:01:25,940 INFO [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-09-03 05:01:25,940 DEBUG [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(1478): Online Regions={d49214d2a25145de199d28b106bafe32=hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32., 1588230740=hbase:meta,,1.1588230740, b3c1671eba7bdf52eda9c2a56f0dc37e=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e.} 2023-09-03 05:01:25,940 DEBUG [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(1504): Waiting on 1588230740, b3c1671eba7bdf52eda9c2a56f0dc37e, d49214d2a25145de199d28b106bafe32 2023-09-03 05:01:25,941 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-03 05:01:25,941 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-03 05:01:25,941 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-03 05:01:25,941 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-03 05:01:25,941 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-03 05:01:25,941 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.10 KB heapSize=5.61 KB 2023-09-03 05:01:25,946 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/namespace/d49214d2a25145de199d28b106bafe32/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-09-03 05:01:25,947 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. 2023-09-03 05:01:25,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d49214d2a25145de199d28b106bafe32: 2023-09-03 05:01:25,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693717224859.d49214d2a25145de199d28b106bafe32. 2023-09-03 05:01:25,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b3c1671eba7bdf52eda9c2a56f0dc37e, disabling compactions & flushes 2023-09-03 05:01:25,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:01:25,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:01:25,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. after waiting 0 ms 2023-09-03 05:01:25,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:01:25,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing b3c1671eba7bdf52eda9c2a56f0dc37e 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-03 05:01:25,958 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.84 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/.tmp/info/6df6f9b05aa94ac298fb27c87a90d083 2023-09-03 05:01:25,966 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=22 (bloomFilter=true), to=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/.tmp/info/f614d100536c47ec95521989ea5ee327 2023-09-03 05:01:25,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/.tmp/info/f614d100536c47ec95521989ea5ee327 as hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/f614d100536c47ec95521989ea5ee327 2023-09-03 05:01:25,981 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/f614d100536c47ec95521989ea5ee327, entries=1, sequenceid=22, filesize=5.8 K 2023-09-03 05:01:25,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for b3c1671eba7bdf52eda9c2a56f0dc37e in 34ms, sequenceid=22, compaction requested=true 2023-09-03 05:01:25,987 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/b4201e1b9de74d79b35bea026fa6b51d, hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/e1d62ce75de946c0a0e890350332abf9, hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/184f2a50528844529e6519a9d32c1012] to archive 2023-09-03 05:01:25,988 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-09-03 05:01:25,990 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=264 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/.tmp/table/1096bdfc70d74c29a9dcb5315cb9412e 2023-09-03 05:01:25,993 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/b4201e1b9de74d79b35bea026fa6b51d to hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/b4201e1b9de74d79b35bea026fa6b51d 2023-09-03 05:01:25,995 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/e1d62ce75de946c0a0e890350332abf9 to hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/e1d62ce75de946c0a0e890350332abf9 2023-09-03 05:01:25,997 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/184f2a50528844529e6519a9d32c1012 to hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/info/184f2a50528844529e6519a9d32c1012 2023-09-03 05:01:26,000 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/.tmp/info/6df6f9b05aa94ac298fb27c87a90d083 as hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/info/6df6f9b05aa94ac298fb27c87a90d083 2023-09-03 05:01:26,008 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/info/6df6f9b05aa94ac298fb27c87a90d083, entries=20, sequenceid=14, filesize=7.6 K 2023-09-03 05:01:26,010 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/.tmp/table/1096bdfc70d74c29a9dcb5315cb9412e as hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/table/1096bdfc70d74c29a9dcb5315cb9412e 2023-09-03 05:01:26,019 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/table/1096bdfc70d74c29a9dcb5315cb9412e, entries=4, sequenceid=14, filesize=4.9 K 2023-09-03 05:01:26,020 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.10 KB/3174, heapSize ~5.33 KB/5456, currentSize=0 B/0 for 1588230740 in 79ms, sequenceid=14, compaction requested=false 2023-09-03 05:01:26,033 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-09-03 05:01:26,034 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-03 05:01:26,037 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-03 05:01:26,037 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-03 05:01:26,037 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-09-03 05:01:26,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/b3c1671eba7bdf52eda9c2a56f0dc37e/recovered.edits/25.seqid, newMaxSeqId=25, maxSeqId=1 2023-09-03 05:01:26,039 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:01:26,039 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b3c1671eba7bdf52eda9c2a56f0dc37e: 2023-09-03 05:01:26,039 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693717225384.b3c1671eba7bdf52eda9c2a56f0dc37e. 2023-09-03 05:01:26,140 INFO [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42265,1693717224234; all regions closed. 2023-09-03 05:01:26,147 DEBUG [RS:0;jenkins-hbase4:42265] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/oldWALs 2023-09-03 05:01:26,147 INFO [RS:0;jenkins-hbase4:42265] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42265%2C1693717224234.meta:.meta(num 1693717224793) 2023-09-03 05:01:26,154 DEBUG [RS:0;jenkins-hbase4:42265] wal.AbstractFSWAL(1028): Moved 2 WAL file(s) to /user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/oldWALs 2023-09-03 05:01:26,154 INFO [RS:0;jenkins-hbase4:42265] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42265%2C1693717224234:(num 1693717285914) 2023-09-03 05:01:26,154 DEBUG [RS:0;jenkins-hbase4:42265] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:01:26,154 INFO [RS:0;jenkins-hbase4:42265] regionserver.LeaseManager(133): Closed leases 2023-09-03 05:01:26,154 INFO [RS:0;jenkins-hbase4:42265] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-09-03 05:01:26,154 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-03 05:01:26,155 INFO [RS:0;jenkins-hbase4:42265] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42265 2023-09-03 05:01:26,158 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-03 05:01:26,158 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42265,1693717224234 2023-09-03 05:01:26,158 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-03 05:01:26,158 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42265,1693717224234] 2023-09-03 05:01:26,158 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42265,1693717224234; numProcessing=1 2023-09-03 05:01:26,161 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42265,1693717224234 already deleted, retry=false 2023-09-03 05:01:26,161 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42265,1693717224234 expired; onlineServers=0 2023-09-03 05:01:26,161 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45873,1693717224179' ***** 2023-09-03 05:01:26,161 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-09-03 05:01:26,161 DEBUG [M:0;jenkins-hbase4:45873] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2b2850ac, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-03 05:01:26,162 INFO [M:0;jenkins-hbase4:45873] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45873,1693717224179 2023-09-03 05:01:26,162 INFO [M:0;jenkins-hbase4:45873] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45873,1693717224179; all regions closed. 2023-09-03 05:01:26,162 DEBUG [M:0;jenkins-hbase4:45873] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:01:26,162 DEBUG [M:0;jenkins-hbase4:45873] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-09-03 05:01:26,162 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-09-03 05:01:26,162 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693717224413] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693717224413,5,FailOnTimeoutGroup] 2023-09-03 05:01:26,162 DEBUG [M:0;jenkins-hbase4:45873] cleaner.HFileCleaner(317): Stopping file delete threads 2023-09-03 05:01:26,162 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693717224412] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693717224412,5,FailOnTimeoutGroup] 2023-09-03 05:01:26,162 INFO [M:0;jenkins-hbase4:45873] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-09-03 05:01:26,163 INFO [M:0;jenkins-hbase4:45873] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-09-03 05:01:26,163 INFO [M:0;jenkins-hbase4:45873] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-09-03 05:01:26,163 DEBUG [M:0;jenkins-hbase4:45873] master.HMaster(1512): Stopping service threads 2023-09-03 05:01:26,163 INFO [M:0;jenkins-hbase4:45873] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-09-03 05:01:26,163 ERROR [M:0;jenkins-hbase4:45873] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-09-03 05:01:26,163 INFO [M:0;jenkins-hbase4:45873] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-09-03 05:01:26,163 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-09-03 05:01:26,163 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:01:26,164 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-09-03 05:01:26,164 DEBUG [M:0;jenkins-hbase4:45873] zookeeper.ZKUtil(398): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-09-03 05:01:26,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-03 05:01:26,164 WARN [M:0;jenkins-hbase4:45873] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-09-03 05:01:26,164 INFO [M:0;jenkins-hbase4:45873] assignment.AssignmentManager(315): Stopping assignment manager 2023-09-03 05:01:26,164 INFO [M:0;jenkins-hbase4:45873] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-09-03 05:01:26,165 DEBUG [M:0;jenkins-hbase4:45873] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-03 05:01:26,165 INFO [M:0;jenkins-hbase4:45873] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:01:26,165 DEBUG [M:0;jenkins-hbase4:45873] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:01:26,165 DEBUG [M:0;jenkins-hbase4:45873] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-03 05:01:26,165 DEBUG [M:0;jenkins-hbase4:45873] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:01:26,165 INFO [M:0;jenkins-hbase4:45873] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.89 KB heapSize=47.33 KB 2023-09-03 05:01:26,179 INFO [M:0;jenkins-hbase4:45873] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.89 KB at sequenceid=100 (bloomFilter=true), to=hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d654355ee1894113a77dca48fea8b0e0 2023-09-03 05:01:26,185 INFO [M:0;jenkins-hbase4:45873] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d654355ee1894113a77dca48fea8b0e0 2023-09-03 05:01:26,186 DEBUG [M:0;jenkins-hbase4:45873] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d654355ee1894113a77dca48fea8b0e0 as hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d654355ee1894113a77dca48fea8b0e0 2023-09-03 05:01:26,192 INFO [M:0;jenkins-hbase4:45873] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d654355ee1894113a77dca48fea8b0e0 2023-09-03 05:01:26,192 INFO [M:0;jenkins-hbase4:45873] regionserver.HStore(1080): Added hdfs://localhost:45657/user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d654355ee1894113a77dca48fea8b0e0, entries=11, sequenceid=100, filesize=6.1 K 2023-09-03 05:01:26,193 INFO [M:0;jenkins-hbase4:45873] regionserver.HRegion(2948): Finished flush of dataSize ~38.89 KB/39824, heapSize ~47.31 KB/48448, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 28ms, sequenceid=100, compaction requested=false 2023-09-03 05:01:26,194 INFO [M:0;jenkins-hbase4:45873] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:01:26,194 DEBUG [M:0;jenkins-hbase4:45873] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-03 05:01:26,252 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/fa29d16a-386c-42fc-7874-669861fce849/MasterData/WALs/jenkins-hbase4.apache.org,45873,1693717224179/jenkins-hbase4.apache.org%2C45873%2C1693717224179.1693717224327 not finished, retry = 0 2023-09-03 05:01:26,260 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:01:26,260 INFO [RS:0;jenkins-hbase4:42265] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42265,1693717224234; zookeeper connection closed. 2023-09-03 05:01:26,261 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): regionserver:42265-0x100702912850001, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:01:26,261 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@47d45561] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@47d45561 2023-09-03 05:01:26,261 INFO [Listener at localhost/40333] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-09-03 05:01:26,353 INFO [M:0;jenkins-hbase4:45873] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-09-03 05:01:26,353 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-03 05:01:26,354 INFO [M:0;jenkins-hbase4:45873] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45873 2023-09-03 05:01:26,356 DEBUG [M:0;jenkins-hbase4:45873] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,45873,1693717224179 already deleted, retry=false 2023-09-03 05:01:26,458 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:01:26,458 INFO [M:0;jenkins-hbase4:45873] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45873,1693717224179; zookeeper connection closed. 2023-09-03 05:01:26,459 DEBUG [Listener at localhost/40333-EventThread] zookeeper.ZKWatcher(600): master:45873-0x100702912850000, quorum=127.0.0.1:57996, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:01:26,460 WARN [Listener at localhost/40333] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-03 05:01:26,463 INFO [Listener at localhost/40333] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:01:26,503 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-09-03 05:01:26,566 WARN [BP-1475551041-172.31.14.131-1693717223596 heartbeating to localhost/127.0.0.1:45657] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-03 05:01:26,566 WARN [BP-1475551041-172.31.14.131-1693717223596 heartbeating to localhost/127.0.0.1:45657] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1475551041-172.31.14.131-1693717223596 (Datanode Uuid 9eeecc76-a237-4311-bde6-01c24676dac7) service to localhost/127.0.0.1:45657 2023-09-03 05:01:26,568 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/cluster_fce263f4-3716-36b0-1730-b29470dde46f/dfs/data/data3/current/BP-1475551041-172.31.14.131-1693717223596] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:01:26,569 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/cluster_fce263f4-3716-36b0-1730-b29470dde46f/dfs/data/data4/current/BP-1475551041-172.31.14.131-1693717223596] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:01:26,570 WARN [Listener at localhost/40333] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-03 05:01:26,576 INFO [Listener at localhost/40333] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:01:26,679 WARN [BP-1475551041-172.31.14.131-1693717223596 heartbeating to localhost/127.0.0.1:45657] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-03 05:01:26,679 WARN [BP-1475551041-172.31.14.131-1693717223596 heartbeating to localhost/127.0.0.1:45657] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1475551041-172.31.14.131-1693717223596 (Datanode Uuid 5fa208d1-5240-457d-b878-1155bdc696fb) service to localhost/127.0.0.1:45657 2023-09-03 05:01:26,680 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/cluster_fce263f4-3716-36b0-1730-b29470dde46f/dfs/data/data1/current/BP-1475551041-172.31.14.131-1693717223596] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:01:26,680 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/cluster_fce263f4-3716-36b0-1730-b29470dde46f/dfs/data/data2/current/BP-1475551041-172.31.14.131-1693717223596] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:01:26,691 INFO [Listener at localhost/40333] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:01:26,804 INFO [Listener at localhost/40333] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-09-03 05:01:26,820 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-09-03 05:01:26,829 INFO [Listener at localhost/40333] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=63 (was 78), OpenFileDescriptor=433 (was 467), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=93 (was 198), ProcessCount=167 (was 167), AvailableMemoryMB=8356 (was 8624) 2023-09-03 05:01:26,838 INFO [Listener at localhost/40333] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=64, OpenFileDescriptor=433, MaxFileDescriptor=60000, SystemLoadAverage=93, ProcessCount=167, AvailableMemoryMB=8356 2023-09-03 05:01:26,838 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-09-03 05:01:26,839 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/hadoop.log.dir so I do NOT create it in target/test-data/896df6f1-906b-f42a-a479-324b101022d7 2023-09-03 05:01:26,839 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a9d771e-9387-a96e-a4fc-d9cafb0124f0/hadoop.tmp.dir so I do NOT create it in target/test-data/896df6f1-906b-f42a-a479-324b101022d7 2023-09-03 05:01:26,839 INFO [Listener at localhost/40333] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/cluster_570609d1-7cf6-2009-b5e0-6d4de4b161ae, deleteOnExit=true 2023-09-03 05:01:26,839 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-09-03 05:01:26,839 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/test.cache.data in system properties and HBase conf 2023-09-03 05:01:26,839 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/hadoop.tmp.dir in system properties and HBase conf 2023-09-03 05:01:26,839 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/hadoop.log.dir in system properties and HBase conf 2023-09-03 05:01:26,839 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/mapreduce.cluster.local.dir in system properties and HBase conf 2023-09-03 05:01:26,839 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-09-03 05:01:26,839 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-09-03 05:01:26,840 DEBUG [Listener at localhost/40333] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-09-03 05:01:26,840 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-09-03 05:01:26,840 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-09-03 05:01:26,840 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-09-03 05:01:26,840 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-03 05:01:26,840 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-09-03 05:01:26,840 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-09-03 05:01:26,840 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-03 05:01:26,841 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-03 05:01:26,841 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-09-03 05:01:26,841 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/nfs.dump.dir in system properties and HBase conf 2023-09-03 05:01:26,841 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/java.io.tmpdir in system properties and HBase conf 2023-09-03 05:01:26,841 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-03 05:01:26,841 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-09-03 05:01:26,841 INFO [Listener at localhost/40333] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-09-03 05:01:26,845 WARN [Listener at localhost/40333] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-03 05:01:26,846 WARN [Listener at localhost/40333] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-03 05:01:26,890 WARN [Listener at localhost/40333] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 05:01:26,893 INFO [Listener at localhost/40333] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 05:01:26,898 INFO [Listener at localhost/40333] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/java.io.tmpdir/Jetty_localhost_40995_hdfs____okcts3/webapp 2023-09-03 05:01:26,989 INFO [Listener at localhost/40333] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40995 2023-09-03 05:01:26,994 WARN [Listener at localhost/40333] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-03 05:01:26,994 WARN [Listener at localhost/40333] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-03 05:01:27,031 WARN [Listener at localhost/39747] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 05:01:27,041 WARN [Listener at localhost/39747] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-03 05:01:27,043 WARN [Listener at localhost/39747] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 05:01:27,044 INFO [Listener at localhost/39747] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 05:01:27,048 INFO [Listener at localhost/39747] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/java.io.tmpdir/Jetty_localhost_37891_datanode____.k0pui5/webapp 2023-09-03 05:01:27,139 INFO [Listener at localhost/39747] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37891 2023-09-03 05:01:27,145 WARN [Listener at localhost/37971] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 05:01:27,157 WARN [Listener at localhost/37971] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-03 05:01:27,158 WARN [Listener at localhost/37971] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 05:01:27,159 INFO [Listener at localhost/37971] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 05:01:27,163 INFO [Listener at localhost/37971] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/java.io.tmpdir/Jetty_localhost_41005_datanode____.jkcsch/webapp 2023-09-03 05:01:27,237 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x494688600ed278cc: Processing first storage report for DS-064a3f45-f12d-4222-996a-fd42de53fd59 from datanode ac4a592c-517b-4ef9-9ce4-16f8ab144043 2023-09-03 05:01:27,237 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x494688600ed278cc: from storage DS-064a3f45-f12d-4222-996a-fd42de53fd59 node DatanodeRegistration(127.0.0.1:37087, datanodeUuid=ac4a592c-517b-4ef9-9ce4-16f8ab144043, infoPort=46173, infoSecurePort=0, ipcPort=37971, storageInfo=lv=-57;cid=testClusterID;nsid=105917237;c=1693717286848), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:01:27,237 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x494688600ed278cc: Processing first storage report for DS-0220469b-86be-48d9-8c8c-9b1b0ce734f3 from datanode ac4a592c-517b-4ef9-9ce4-16f8ab144043 2023-09-03 05:01:27,237 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x494688600ed278cc: from storage DS-0220469b-86be-48d9-8c8c-9b1b0ce734f3 node DatanodeRegistration(127.0.0.1:37087, datanodeUuid=ac4a592c-517b-4ef9-9ce4-16f8ab144043, infoPort=46173, infoSecurePort=0, ipcPort=37971, storageInfo=lv=-57;cid=testClusterID;nsid=105917237;c=1693717286848), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:01:27,257 INFO [Listener at localhost/37971] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41005 2023-09-03 05:01:27,263 WARN [Listener at localhost/45463] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 05:01:27,353 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x5de6eea90cb377ee: Processing first storage report for DS-eae5636d-4bd8-4d07-9229-04293ad8a469 from datanode e27a9e9b-c163-4ef4-bc88-e927d7f3b165 2023-09-03 05:01:27,353 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x5de6eea90cb377ee: from storage DS-eae5636d-4bd8-4d07-9229-04293ad8a469 node DatanodeRegistration(127.0.0.1:42837, datanodeUuid=e27a9e9b-c163-4ef4-bc88-e927d7f3b165, infoPort=38303, infoSecurePort=0, ipcPort=45463, storageInfo=lv=-57;cid=testClusterID;nsid=105917237;c=1693717286848), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:01:27,353 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x5de6eea90cb377ee: Processing first storage report for DS-897260de-5919-4ece-8911-c109aa5736a1 from datanode e27a9e9b-c163-4ef4-bc88-e927d7f3b165 2023-09-03 05:01:27,353 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x5de6eea90cb377ee: from storage DS-897260de-5919-4ece-8911-c109aa5736a1 node DatanodeRegistration(127.0.0.1:42837, datanodeUuid=e27a9e9b-c163-4ef4-bc88-e927d7f3b165, infoPort=38303, infoSecurePort=0, ipcPort=45463, storageInfo=lv=-57;cid=testClusterID;nsid=105917237;c=1693717286848), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:01:27,370 DEBUG [Listener at localhost/45463] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7 2023-09-03 05:01:27,372 INFO [Listener at localhost/45463] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/cluster_570609d1-7cf6-2009-b5e0-6d4de4b161ae/zookeeper_0, clientPort=53078, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/cluster_570609d1-7cf6-2009-b5e0-6d4de4b161ae/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/cluster_570609d1-7cf6-2009-b5e0-6d4de4b161ae/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-09-03 05:01:27,373 INFO [Listener at localhost/45463] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=53078 2023-09-03 05:01:27,374 INFO [Listener at localhost/45463] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:01:27,374 INFO [Listener at localhost/45463] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:01:27,388 INFO [Listener at localhost/45463] util.FSUtils(471): Created version file at hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d with version=8 2023-09-03 05:01:27,388 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/hbase-staging 2023-09-03 05:01:27,389 INFO [Listener at localhost/45463] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-09-03 05:01:27,390 INFO [Listener at localhost/45463] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:01:27,390 INFO [Listener at localhost/45463] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-03 05:01:27,390 INFO [Listener at localhost/45463] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-03 05:01:27,390 INFO [Listener at localhost/45463] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:01:27,390 INFO [Listener at localhost/45463] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-03 05:01:27,390 INFO [Listener at localhost/45463] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-09-03 05:01:27,391 INFO [Listener at localhost/45463] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39041 2023-09-03 05:01:27,392 INFO [Listener at localhost/45463] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:01:27,393 INFO [Listener at localhost/45463] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:01:27,394 INFO [Listener at localhost/45463] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39041 connecting to ZooKeeper ensemble=127.0.0.1:53078 2023-09-03 05:01:27,406 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:390410x0, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-03 05:01:27,407 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:39041-0x100702a09720000 connected 2023-09-03 05:01:27,426 DEBUG [Listener at localhost/45463] zookeeper.ZKUtil(164): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-03 05:01:27,426 DEBUG [Listener at localhost/45463] zookeeper.ZKUtil(164): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:01:27,427 DEBUG [Listener at localhost/45463] zookeeper.ZKUtil(164): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-03 05:01:27,427 DEBUG [Listener at localhost/45463] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39041 2023-09-03 05:01:27,427 DEBUG [Listener at localhost/45463] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39041 2023-09-03 05:01:27,428 DEBUG [Listener at localhost/45463] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39041 2023-09-03 05:01:27,428 DEBUG [Listener at localhost/45463] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39041 2023-09-03 05:01:27,428 DEBUG [Listener at localhost/45463] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39041 2023-09-03 05:01:27,428 INFO [Listener at localhost/45463] master.HMaster(444): hbase.rootdir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d, hbase.cluster.distributed=false 2023-09-03 05:01:27,441 INFO [Listener at localhost/45463] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-09-03 05:01:27,441 INFO [Listener at localhost/45463] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:01:27,441 INFO [Listener at localhost/45463] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-03 05:01:27,441 INFO [Listener at localhost/45463] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-03 05:01:27,442 INFO [Listener at localhost/45463] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:01:27,442 INFO [Listener at localhost/45463] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-03 05:01:27,442 INFO [Listener at localhost/45463] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-09-03 05:01:27,443 INFO [Listener at localhost/45463] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35525 2023-09-03 05:01:27,443 INFO [Listener at localhost/45463] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-03 05:01:27,444 DEBUG [Listener at localhost/45463] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-09-03 05:01:27,444 INFO [Listener at localhost/45463] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:01:27,445 INFO [Listener at localhost/45463] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:01:27,446 INFO [Listener at localhost/45463] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:35525 connecting to ZooKeeper ensemble=127.0.0.1:53078 2023-09-03 05:01:27,449 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): regionserver:355250x0, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-03 05:01:27,450 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:35525-0x100702a09720001 connected 2023-09-03 05:01:27,450 DEBUG [Listener at localhost/45463] zookeeper.ZKUtil(164): regionserver:35525-0x100702a09720001, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-03 05:01:27,450 DEBUG [Listener at localhost/45463] zookeeper.ZKUtil(164): regionserver:35525-0x100702a09720001, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:01:27,451 DEBUG [Listener at localhost/45463] zookeeper.ZKUtil(164): regionserver:35525-0x100702a09720001, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-03 05:01:27,451 DEBUG [Listener at localhost/45463] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35525 2023-09-03 05:01:27,451 DEBUG [Listener at localhost/45463] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35525 2023-09-03 05:01:27,451 DEBUG [Listener at localhost/45463] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35525 2023-09-03 05:01:27,452 DEBUG [Listener at localhost/45463] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35525 2023-09-03 05:01:27,452 DEBUG [Listener at localhost/45463] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35525 2023-09-03 05:01:27,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,39041,1693717287389 2023-09-03 05:01:27,454 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-03 05:01:27,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,39041,1693717287389 2023-09-03 05:01:27,456 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): regionserver:35525-0x100702a09720001, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-03 05:01:27,456 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-03 05:01:27,456 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:01:27,457 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-03 05:01:27,457 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-03 05:01:27,457 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,39041,1693717287389 from backup master directory 2023-09-03 05:01:27,459 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,39041,1693717287389 2023-09-03 05:01:27,459 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-03 05:01:27,459 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-03 05:01:27,460 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,39041,1693717287389 2023-09-03 05:01:27,473 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/hbase.id with ID: 68391994-4f85-4060-a718-a691ce8ae3cd 2023-09-03 05:01:27,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:01:27,486 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:01:27,496 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x1769a049 to 127.0.0.1:53078 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-03 05:01:27,499 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4ebc4199, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-03 05:01:27,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-03 05:01:27,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-09-03 05:01:27,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-03 05:01:27,502 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/MasterData/data/master/store-tmp 2023-09-03 05:01:27,512 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:01:27,512 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-03 05:01:27,512 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:01:27,512 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:01:27,513 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-03 05:01:27,513 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:01:27,513 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:01:27,513 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-03 05:01:27,513 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/MasterData/WALs/jenkins-hbase4.apache.org,39041,1693717287389 2023-09-03 05:01:27,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39041%2C1693717287389, suffix=, logDir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/MasterData/WALs/jenkins-hbase4.apache.org,39041,1693717287389, archiveDir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/MasterData/oldWALs, maxLogs=10 2023-09-03 05:01:27,537 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42837,DS-eae5636d-4bd8-4d07-9229-04293ad8a469,DISK] 2023-09-03 05:01:27,537 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37087,DS-064a3f45-f12d-4222-996a-fd42de53fd59,DISK] 2023-09-03 05:01:27,540 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/MasterData/WALs/jenkins-hbase4.apache.org,39041,1693717287389/jenkins-hbase4.apache.org%2C39041%2C1693717287389.1693717287517 2023-09-03 05:01:27,540 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42837,DS-eae5636d-4bd8-4d07-9229-04293ad8a469,DISK], DatanodeInfoWithStorage[127.0.0.1:37087,DS-064a3f45-f12d-4222-996a-fd42de53fd59,DISK]] 2023-09-03 05:01:27,540 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-09-03 05:01:27,540 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:01:27,540 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:01:27,540 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:01:27,542 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:01:27,543 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-09-03 05:01:27,544 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-09-03 05:01:27,544 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:01:27,545 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:01:27,546 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:01:27,548 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:01:27,550 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-03 05:01:27,550 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=715437, jitterRate=-0.09027500450611115}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-03 05:01:27,551 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-03 05:01:27,553 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-09-03 05:01:27,554 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-09-03 05:01:27,554 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-09-03 05:01:27,554 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-09-03 05:01:27,554 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-09-03 05:01:27,555 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 0 msec 2023-09-03 05:01:27,555 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-09-03 05:01:27,555 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-09-03 05:01:27,556 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-09-03 05:01:27,567 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-09-03 05:01:27,567 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-09-03 05:01:27,567 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-09-03 05:01:27,567 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-09-03 05:01:27,568 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-09-03 05:01:27,569 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:01:27,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-09-03 05:01:27,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-09-03 05:01:27,571 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-09-03 05:01:27,574 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-03 05:01:27,574 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): regionserver:35525-0x100702a09720001, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-03 05:01:27,574 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:01:27,574 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,39041,1693717287389, sessionid=0x100702a09720000, setting cluster-up flag (Was=false) 2023-09-03 05:01:27,577 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:01:27,581 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-09-03 05:01:27,582 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39041,1693717287389 2023-09-03 05:01:27,584 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:01:27,588 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-09-03 05:01:27,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39041,1693717287389 2023-09-03 05:01:27,590 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/.hbase-snapshot/.tmp 2023-09-03 05:01:27,592 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-09-03 05:01:27,592 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:01:27,592 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:01:27,592 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:01:27,592 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:01:27,592 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-09-03 05:01:27,592 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:01:27,592 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-03 05:01:27,592 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:01:27,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693717317594 2023-09-03 05:01:27,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-09-03 05:01:27,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-09-03 05:01:27,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-09-03 05:01:27,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-09-03 05:01:27,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-09-03 05:01:27,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-09-03 05:01:27,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:27,595 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-09-03 05:01:27,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-09-03 05:01:27,595 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-09-03 05:01:27,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-09-03 05:01:27,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-09-03 05:01:27,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-09-03 05:01:27,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-09-03 05:01:27,596 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693717287596,5,FailOnTimeoutGroup] 2023-09-03 05:01:27,596 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693717287596,5,FailOnTimeoutGroup] 2023-09-03 05:01:27,596 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:27,596 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-09-03 05:01:27,596 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:27,596 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:27,596 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-03 05:01:27,607 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-03 05:01:27,607 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-03 05:01:27,607 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d 2023-09-03 05:01:27,615 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:01:27,617 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-03 05:01:27,618 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/info 2023-09-03 05:01:27,619 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-03 05:01:27,619 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:01:27,619 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-03 05:01:27,620 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/rep_barrier 2023-09-03 05:01:27,621 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-03 05:01:27,621 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:01:27,621 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-03 05:01:27,623 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/table 2023-09-03 05:01:27,623 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-03 05:01:27,623 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:01:27,624 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740 2023-09-03 05:01:27,625 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740 2023-09-03 05:01:27,627 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-03 05:01:27,628 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-03 05:01:27,630 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-03 05:01:27,631 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=855627, jitterRate=0.08798696100711823}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-03 05:01:27,631 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-03 05:01:27,631 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-03 05:01:27,631 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-03 05:01:27,631 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-03 05:01:27,631 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-03 05:01:27,631 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-03 05:01:27,632 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-03 05:01:27,632 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-03 05:01:27,633 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-09-03 05:01:27,633 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-09-03 05:01:27,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-09-03 05:01:27,634 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-09-03 05:01:27,635 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-09-03 05:01:27,654 INFO [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(951): ClusterId : 68391994-4f85-4060-a718-a691ce8ae3cd 2023-09-03 05:01:27,654 DEBUG [RS:0;jenkins-hbase4:35525] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-09-03 05:01:27,656 DEBUG [RS:0;jenkins-hbase4:35525] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-09-03 05:01:27,656 DEBUG [RS:0;jenkins-hbase4:35525] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-09-03 05:01:27,660 DEBUG [RS:0;jenkins-hbase4:35525] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-09-03 05:01:27,661 DEBUG [RS:0;jenkins-hbase4:35525] zookeeper.ReadOnlyZKClient(139): Connect 0x42e5ad83 to 127.0.0.1:53078 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-03 05:01:27,664 DEBUG [RS:0;jenkins-hbase4:35525] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4c225baf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-03 05:01:27,664 DEBUG [RS:0;jenkins-hbase4:35525] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@481dc6f3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-03 05:01:27,673 DEBUG [RS:0;jenkins-hbase4:35525] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:35525 2023-09-03 05:01:27,673 INFO [RS:0;jenkins-hbase4:35525] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-09-03 05:01:27,673 INFO [RS:0;jenkins-hbase4:35525] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-09-03 05:01:27,673 DEBUG [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(1022): About to register with Master. 2023-09-03 05:01:27,674 INFO [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39041,1693717287389 with isa=jenkins-hbase4.apache.org/172.31.14.131:35525, startcode=1693717287441 2023-09-03 05:01:27,674 DEBUG [RS:0;jenkins-hbase4:35525] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-09-03 05:01:27,676 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46745, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-09-03 05:01:27,677 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39041] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:27,677 DEBUG [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d 2023-09-03 05:01:27,678 DEBUG [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:39747 2023-09-03 05:01:27,678 DEBUG [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-09-03 05:01:27,679 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-03 05:01:27,679 DEBUG [RS:0;jenkins-hbase4:35525] zookeeper.ZKUtil(162): regionserver:35525-0x100702a09720001, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:27,680 WARN [RS:0;jenkins-hbase4:35525] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-03 05:01:27,680 INFO [RS:0;jenkins-hbase4:35525] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-03 05:01:27,680 DEBUG [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(1948): logDir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/WALs/jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:27,680 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,35525,1693717287441] 2023-09-03 05:01:27,683 DEBUG [RS:0;jenkins-hbase4:35525] zookeeper.ZKUtil(162): regionserver:35525-0x100702a09720001, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:27,684 DEBUG [RS:0;jenkins-hbase4:35525] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-09-03 05:01:27,684 INFO [RS:0;jenkins-hbase4:35525] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-09-03 05:01:27,685 INFO [RS:0;jenkins-hbase4:35525] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-09-03 05:01:27,693 INFO [RS:0;jenkins-hbase4:35525] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-09-03 05:01:27,693 INFO [RS:0;jenkins-hbase4:35525] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:27,693 INFO [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-09-03 05:01:27,694 INFO [RS:0;jenkins-hbase4:35525] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:27,694 DEBUG [RS:0;jenkins-hbase4:35525] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:01:27,694 DEBUG [RS:0;jenkins-hbase4:35525] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:01:27,694 DEBUG [RS:0;jenkins-hbase4:35525] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:01:27,694 DEBUG [RS:0;jenkins-hbase4:35525] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:01:27,694 DEBUG [RS:0;jenkins-hbase4:35525] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:01:27,694 DEBUG [RS:0;jenkins-hbase4:35525] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-03 05:01:27,694 DEBUG [RS:0;jenkins-hbase4:35525] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:01:27,695 DEBUG [RS:0;jenkins-hbase4:35525] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:01:27,695 DEBUG [RS:0;jenkins-hbase4:35525] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:01:27,695 DEBUG [RS:0;jenkins-hbase4:35525] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:01:27,695 INFO [RS:0;jenkins-hbase4:35525] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:27,695 INFO [RS:0;jenkins-hbase4:35525] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:27,695 INFO [RS:0;jenkins-hbase4:35525] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:27,706 INFO [RS:0;jenkins-hbase4:35525] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-09-03 05:01:27,706 INFO [RS:0;jenkins-hbase4:35525] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35525,1693717287441-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:27,716 INFO [RS:0;jenkins-hbase4:35525] regionserver.Replication(203): jenkins-hbase4.apache.org,35525,1693717287441 started 2023-09-03 05:01:27,716 INFO [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,35525,1693717287441, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:35525, sessionid=0x100702a09720001 2023-09-03 05:01:27,716 DEBUG [RS:0;jenkins-hbase4:35525] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-09-03 05:01:27,716 DEBUG [RS:0;jenkins-hbase4:35525] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:27,716 DEBUG [RS:0;jenkins-hbase4:35525] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35525,1693717287441' 2023-09-03 05:01:27,716 DEBUG [RS:0;jenkins-hbase4:35525] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-03 05:01:27,717 DEBUG [RS:0;jenkins-hbase4:35525] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-03 05:01:27,717 DEBUG [RS:0;jenkins-hbase4:35525] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-09-03 05:01:27,717 DEBUG [RS:0;jenkins-hbase4:35525] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-09-03 05:01:27,717 DEBUG [RS:0;jenkins-hbase4:35525] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:27,717 DEBUG [RS:0;jenkins-hbase4:35525] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35525,1693717287441' 2023-09-03 05:01:27,717 DEBUG [RS:0;jenkins-hbase4:35525] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-09-03 05:01:27,718 DEBUG [RS:0;jenkins-hbase4:35525] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-09-03 05:01:27,718 DEBUG [RS:0;jenkins-hbase4:35525] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-09-03 05:01:27,718 INFO [RS:0;jenkins-hbase4:35525] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-09-03 05:01:27,718 INFO [RS:0;jenkins-hbase4:35525] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-09-03 05:01:27,785 DEBUG [jenkins-hbase4:39041] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-09-03 05:01:27,786 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,35525,1693717287441, state=OPENING 2023-09-03 05:01:27,788 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-09-03 05:01:27,789 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:01:27,790 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,35525,1693717287441}] 2023-09-03 05:01:27,790 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-03 05:01:27,820 INFO [RS:0;jenkins-hbase4:35525] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35525%2C1693717287441, suffix=, logDir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/WALs/jenkins-hbase4.apache.org,35525,1693717287441, archiveDir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/oldWALs, maxLogs=32 2023-09-03 05:01:27,836 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37087,DS-064a3f45-f12d-4222-996a-fd42de53fd59,DISK] 2023-09-03 05:01:27,836 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42837,DS-eae5636d-4bd8-4d07-9229-04293ad8a469,DISK] 2023-09-03 05:01:27,839 INFO [RS:0;jenkins-hbase4:35525] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/WALs/jenkins-hbase4.apache.org,35525,1693717287441/jenkins-hbase4.apache.org%2C35525%2C1693717287441.1693717287821 2023-09-03 05:01:27,839 DEBUG [RS:0;jenkins-hbase4:35525] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37087,DS-064a3f45-f12d-4222-996a-fd42de53fd59,DISK], DatanodeInfoWithStorage[127.0.0.1:42837,DS-eae5636d-4bd8-4d07-9229-04293ad8a469,DISK]] 2023-09-03 05:01:27,944 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:27,944 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-09-03 05:01:27,947 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41700, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-09-03 05:01:27,951 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-09-03 05:01:27,951 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-03 05:01:27,953 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35525%2C1693717287441.meta, suffix=.meta, logDir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/WALs/jenkins-hbase4.apache.org,35525,1693717287441, archiveDir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/oldWALs, maxLogs=32 2023-09-03 05:01:27,968 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42837,DS-eae5636d-4bd8-4d07-9229-04293ad8a469,DISK] 2023-09-03 05:01:27,968 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37087,DS-064a3f45-f12d-4222-996a-fd42de53fd59,DISK] 2023-09-03 05:01:27,970 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/WALs/jenkins-hbase4.apache.org,35525,1693717287441/jenkins-hbase4.apache.org%2C35525%2C1693717287441.meta.1693717287953.meta 2023-09-03 05:01:27,972 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42837,DS-eae5636d-4bd8-4d07-9229-04293ad8a469,DISK], DatanodeInfoWithStorage[127.0.0.1:37087,DS-064a3f45-f12d-4222-996a-fd42de53fd59,DISK]] 2023-09-03 05:01:27,972 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-09-03 05:01:27,972 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-09-03 05:01:27,973 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-09-03 05:01:27,973 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-09-03 05:01:27,973 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-09-03 05:01:27,973 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:01:27,973 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-09-03 05:01:27,973 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-09-03 05:01:27,974 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-03 05:01:27,975 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/info 2023-09-03 05:01:27,975 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/info 2023-09-03 05:01:27,976 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-03 05:01:27,976 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:01:27,977 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-03 05:01:27,977 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/rep_barrier 2023-09-03 05:01:27,978 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/rep_barrier 2023-09-03 05:01:27,978 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-03 05:01:27,979 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:01:27,979 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-03 05:01:27,980 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/table 2023-09-03 05:01:27,980 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/table 2023-09-03 05:01:27,980 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-03 05:01:27,981 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:01:27,981 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740 2023-09-03 05:01:27,983 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740 2023-09-03 05:01:27,985 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-03 05:01:27,986 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-03 05:01:27,987 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=703284, jitterRate=-0.10572934150695801}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-03 05:01:27,987 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-03 05:01:27,988 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693717287944 2023-09-03 05:01:27,991 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-09-03 05:01:27,992 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-09-03 05:01:27,992 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,35525,1693717287441, state=OPEN 2023-09-03 05:01:27,994 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-09-03 05:01:27,994 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-03 05:01:27,997 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-09-03 05:01:27,997 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,35525,1693717287441 in 204 msec 2023-09-03 05:01:27,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-09-03 05:01:27,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 364 msec 2023-09-03 05:01:28,001 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 409 msec 2023-09-03 05:01:28,001 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693717288001, completionTime=-1 2023-09-03 05:01:28,001 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-09-03 05:01:28,002 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-09-03 05:01:28,004 DEBUG [hconnection-0xe31404f-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-03 05:01:28,006 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41708, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-03 05:01:28,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-09-03 05:01:28,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693717348007 2023-09-03 05:01:28,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693717408007 2023-09-03 05:01:28,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-09-03 05:01:28,012 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39041,1693717287389-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:28,012 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39041,1693717287389-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:28,012 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39041,1693717287389-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:28,012 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:39041, period=300000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:28,012 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-09-03 05:01:28,013 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-09-03 05:01:28,013 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-03 05:01:28,014 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-09-03 05:01:28,014 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-09-03 05:01:28,015 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-09-03 05:01:28,016 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-03 05:01:28,017 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/.tmp/data/hbase/namespace/d0ce8cdf1fe0a3b027b62dc1ee5530a6 2023-09-03 05:01:28,018 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/.tmp/data/hbase/namespace/d0ce8cdf1fe0a3b027b62dc1ee5530a6 empty. 2023-09-03 05:01:28,018 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/.tmp/data/hbase/namespace/d0ce8cdf1fe0a3b027b62dc1ee5530a6 2023-09-03 05:01:28,018 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-09-03 05:01:28,029 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-09-03 05:01:28,030 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => d0ce8cdf1fe0a3b027b62dc1ee5530a6, NAME => 'hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/.tmp 2023-09-03 05:01:28,040 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:01:28,041 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing d0ce8cdf1fe0a3b027b62dc1ee5530a6, disabling compactions & flushes 2023-09-03 05:01:28,041 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6. 2023-09-03 05:01:28,041 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6. 2023-09-03 05:01:28,041 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6. after waiting 0 ms 2023-09-03 05:01:28,041 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6. 2023-09-03 05:01:28,041 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6. 2023-09-03 05:01:28,041 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for d0ce8cdf1fe0a3b027b62dc1ee5530a6: 2023-09-03 05:01:28,043 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-09-03 05:01:28,045 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693717288044"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693717288044"}]},"ts":"1693717288044"} 2023-09-03 05:01:28,047 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-03 05:01:28,048 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-03 05:01:28,048 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693717288048"}]},"ts":"1693717288048"} 2023-09-03 05:01:28,049 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-09-03 05:01:28,056 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d0ce8cdf1fe0a3b027b62dc1ee5530a6, ASSIGN}] 2023-09-03 05:01:28,058 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d0ce8cdf1fe0a3b027b62dc1ee5530a6, ASSIGN 2023-09-03 05:01:28,059 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=d0ce8cdf1fe0a3b027b62dc1ee5530a6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35525,1693717287441; forceNewPlan=false, retain=false 2023-09-03 05:01:28,210 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=d0ce8cdf1fe0a3b027b62dc1ee5530a6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:28,210 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693717288210"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693717288210"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693717288210"}]},"ts":"1693717288210"} 2023-09-03 05:01:28,212 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure d0ce8cdf1fe0a3b027b62dc1ee5530a6, server=jenkins-hbase4.apache.org,35525,1693717287441}] 2023-09-03 05:01:28,368 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6. 2023-09-03 05:01:28,368 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d0ce8cdf1fe0a3b027b62dc1ee5530a6, NAME => 'hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6.', STARTKEY => '', ENDKEY => ''} 2023-09-03 05:01:28,369 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace d0ce8cdf1fe0a3b027b62dc1ee5530a6 2023-09-03 05:01:28,369 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:01:28,369 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d0ce8cdf1fe0a3b027b62dc1ee5530a6 2023-09-03 05:01:28,369 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d0ce8cdf1fe0a3b027b62dc1ee5530a6 2023-09-03 05:01:28,370 INFO [StoreOpener-d0ce8cdf1fe0a3b027b62dc1ee5530a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region d0ce8cdf1fe0a3b027b62dc1ee5530a6 2023-09-03 05:01:28,372 DEBUG [StoreOpener-d0ce8cdf1fe0a3b027b62dc1ee5530a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/namespace/d0ce8cdf1fe0a3b027b62dc1ee5530a6/info 2023-09-03 05:01:28,372 DEBUG [StoreOpener-d0ce8cdf1fe0a3b027b62dc1ee5530a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/namespace/d0ce8cdf1fe0a3b027b62dc1ee5530a6/info 2023-09-03 05:01:28,372 INFO [StoreOpener-d0ce8cdf1fe0a3b027b62dc1ee5530a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d0ce8cdf1fe0a3b027b62dc1ee5530a6 columnFamilyName info 2023-09-03 05:01:28,373 INFO [StoreOpener-d0ce8cdf1fe0a3b027b62dc1ee5530a6-1] regionserver.HStore(310): Store=d0ce8cdf1fe0a3b027b62dc1ee5530a6/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:01:28,373 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/namespace/d0ce8cdf1fe0a3b027b62dc1ee5530a6 2023-09-03 05:01:28,374 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/namespace/d0ce8cdf1fe0a3b027b62dc1ee5530a6 2023-09-03 05:01:28,377 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d0ce8cdf1fe0a3b027b62dc1ee5530a6 2023-09-03 05:01:28,379 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/namespace/d0ce8cdf1fe0a3b027b62dc1ee5530a6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-03 05:01:28,379 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d0ce8cdf1fe0a3b027b62dc1ee5530a6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=757159, jitterRate=-0.03722289204597473}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-03 05:01:28,379 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d0ce8cdf1fe0a3b027b62dc1ee5530a6: 2023-09-03 05:01:28,380 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6., pid=6, masterSystemTime=1693717288365 2023-09-03 05:01:28,382 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6. 2023-09-03 05:01:28,382 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6. 2023-09-03 05:01:28,383 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=d0ce8cdf1fe0a3b027b62dc1ee5530a6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:28,383 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693717288383"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693717288383"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693717288383"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693717288383"}]},"ts":"1693717288383"} 2023-09-03 05:01:28,387 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-09-03 05:01:28,387 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure d0ce8cdf1fe0a3b027b62dc1ee5530a6, server=jenkins-hbase4.apache.org,35525,1693717287441 in 173 msec 2023-09-03 05:01:28,389 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-09-03 05:01:28,390 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=d0ce8cdf1fe0a3b027b62dc1ee5530a6, ASSIGN in 331 msec 2023-09-03 05:01:28,390 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-03 05:01:28,391 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693717288391"}]},"ts":"1693717288391"} 2023-09-03 05:01:28,392 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-09-03 05:01:28,394 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-09-03 05:01:28,395 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 381 msec 2023-09-03 05:01:28,415 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-09-03 05:01:28,416 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-09-03 05:01:28,416 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:01:28,420 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-09-03 05:01:28,428 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-03 05:01:28,431 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 11 msec 2023-09-03 05:01:28,442 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-09-03 05:01:28,450 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-03 05:01:28,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 11 msec 2023-09-03 05:01:28,469 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-09-03 05:01:28,471 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-09-03 05:01:28,471 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.011sec 2023-09-03 05:01:28,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-09-03 05:01:28,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-09-03 05:01:28,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-09-03 05:01:28,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39041,1693717287389-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-09-03 05:01:28,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39041,1693717287389-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-09-03 05:01:28,474 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-09-03 05:01:28,554 DEBUG [Listener at localhost/45463] zookeeper.ReadOnlyZKClient(139): Connect 0x26cadc1d to 127.0.0.1:53078 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-03 05:01:28,561 DEBUG [Listener at localhost/45463] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@24610f2c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-03 05:01:28,562 DEBUG [hconnection-0x369b80bf-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-03 05:01:28,564 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41714, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-03 05:01:28,566 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,39041,1693717287389 2023-09-03 05:01:28,566 INFO [Listener at localhost/45463] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:01:28,569 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-09-03 05:01:28,569 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:01:28,570 INFO [Listener at localhost/45463] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-09-03 05:01:28,572 DEBUG [Listener at localhost/45463] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-09-03 05:01:28,578 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36628, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-09-03 05:01:28,580 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39041] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-09-03 05:01:28,580 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39041] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-09-03 05:01:28,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39041] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-03 05:01:28,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39041] procedure2.ProcedureExecutor(1028): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRolling 2023-09-03 05:01:28,586 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-09-03 05:01:28,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39041] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRolling" procId is: 9 2023-09-03 05:01:28,587 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-03 05:01:28,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39041] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-03 05:01:28,589 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/.tmp/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:28,590 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/.tmp/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae empty. 2023-09-03 05:01:28,590 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/.tmp/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:28,590 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRolling regions 2023-09-03 05:01:28,607 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/.tmp/data/default/TestLogRolling-testLogRolling/.tabledesc/.tableinfo.0000000001 2023-09-03 05:01:28,608 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1215e984d247204ad8c8b77a0af178ae, NAME => 'TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/.tmp 2023-09-03 05:01:28,617 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:01:28,617 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1604): Closing 1215e984d247204ad8c8b77a0af178ae, disabling compactions & flushes 2023-09-03 05:01:28,617 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. 2023-09-03 05:01:28,617 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. 2023-09-03 05:01:28,617 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. after waiting 0 ms 2023-09-03 05:01:28,617 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. 2023-09-03 05:01:28,617 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. 2023-09-03 05:01:28,618 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1558): Region close journal for 1215e984d247204ad8c8b77a0af178ae: 2023-09-03 05:01:28,620 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ADD_TO_META 2023-09-03 05:01:28,622 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693717288622"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693717288622"}]},"ts":"1693717288622"} 2023-09-03 05:01:28,624 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-03 05:01:28,625 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-03 05:01:28,625 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693717288625"}]},"ts":"1693717288625"} 2023-09-03 05:01:28,626 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLING in hbase:meta 2023-09-03 05:01:28,634 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=1215e984d247204ad8c8b77a0af178ae, ASSIGN}] 2023-09-03 05:01:28,635 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=1215e984d247204ad8c8b77a0af178ae, ASSIGN 2023-09-03 05:01:28,636 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=1215e984d247204ad8c8b77a0af178ae, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35525,1693717287441; forceNewPlan=false, retain=false 2023-09-03 05:01:28,787 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=1215e984d247204ad8c8b77a0af178ae, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:28,787 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693717288787"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693717288787"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693717288787"}]},"ts":"1693717288787"} 2023-09-03 05:01:28,789 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 1215e984d247204ad8c8b77a0af178ae, server=jenkins-hbase4.apache.org,35525,1693717287441}] 2023-09-03 05:01:28,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. 2023-09-03 05:01:28,946 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1215e984d247204ad8c8b77a0af178ae, NAME => 'TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.', STARTKEY => '', ENDKEY => ''} 2023-09-03 05:01:28,946 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:28,946 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:01:28,946 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:28,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:28,948 INFO [StoreOpener-1215e984d247204ad8c8b77a0af178ae-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:28,949 DEBUG [StoreOpener-1215e984d247204ad8c8b77a0af178ae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info 2023-09-03 05:01:28,950 DEBUG [StoreOpener-1215e984d247204ad8c8b77a0af178ae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info 2023-09-03 05:01:28,950 INFO [StoreOpener-1215e984d247204ad8c8b77a0af178ae-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1215e984d247204ad8c8b77a0af178ae columnFamilyName info 2023-09-03 05:01:28,951 INFO [StoreOpener-1215e984d247204ad8c8b77a0af178ae-1] regionserver.HStore(310): Store=1215e984d247204ad8c8b77a0af178ae/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:01:28,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:28,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:28,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:28,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-03 05:01:28,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1215e984d247204ad8c8b77a0af178ae; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=800342, jitterRate=0.017687484622001648}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-03 05:01:28,959 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1215e984d247204ad8c8b77a0af178ae: 2023-09-03 05:01:28,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae., pid=11, masterSystemTime=1693717288941 2023-09-03 05:01:28,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. 2023-09-03 05:01:28,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. 2023-09-03 05:01:28,962 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=1215e984d247204ad8c8b77a0af178ae, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:28,963 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693717288962"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693717288962"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693717288962"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693717288962"}]},"ts":"1693717288962"} 2023-09-03 05:01:28,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=11, resume processing ppid=10 2023-09-03 05:01:28,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 1215e984d247204ad8c8b77a0af178ae, server=jenkins-hbase4.apache.org,35525,1693717287441 in 176 msec 2023-09-03 05:01:28,969 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=10, resume processing ppid=9 2023-09-03 05:01:28,970 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=1215e984d247204ad8c8b77a0af178ae, ASSIGN in 334 msec 2023-09-03 05:01:28,971 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-03 05:01:28,971 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693717288971"}]},"ts":"1693717288971"} 2023-09-03 05:01:28,972 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLED in hbase:meta 2023-09-03 05:01:28,974 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_POST_OPERATION 2023-09-03 05:01:28,976 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRolling in 394 msec 2023-09-03 05:01:31,476 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-03 05:01:33,684 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-09-03 05:01:33,685 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-09-03 05:01:33,685 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRolling' 2023-09-03 05:01:38,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39041] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-03 05:01:38,589 INFO [Listener at localhost/45463] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRolling, procId: 9 completed 2023-09-03 05:01:38,592 DEBUG [Listener at localhost/45463] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRolling 2023-09-03 05:01:38,593 DEBUG [Listener at localhost/45463] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. 2023-09-03 05:01:38,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:38,607 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1215e984d247204ad8c8b77a0af178ae 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-03 05:01:38,621 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=12 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/f0a5e10f7f464716b9e0d343bee1a67d 2023-09-03 05:01:38,631 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/f0a5e10f7f464716b9e0d343bee1a67d as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/f0a5e10f7f464716b9e0d343bee1a67d 2023-09-03 05:01:38,638 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/f0a5e10f7f464716b9e0d343bee1a67d, entries=8, sequenceid=12, filesize=13.0 K 2023-09-03 05:01:38,639 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=20.53 KB/21020 for 1215e984d247204ad8c8b77a0af178ae in 32ms, sequenceid=12, compaction requested=false 2023-09-03 05:01:38,640 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1215e984d247204ad8c8b77a0af178ae: 2023-09-03 05:01:38,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:38,640 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1215e984d247204ad8c8b77a0af178ae 1/1 column families, dataSize=21.55 KB heapSize=23.38 KB 2023-09-03 05:01:38,654 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=21.55 KB at sequenceid=36 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/131f2c5d082941cbbf3daca8507e2e7d 2023-09-03 05:01:38,660 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/131f2c5d082941cbbf3daca8507e2e7d as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/131f2c5d082941cbbf3daca8507e2e7d 2023-09-03 05:01:38,665 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/131f2c5d082941cbbf3daca8507e2e7d, entries=21, sequenceid=36, filesize=26.3 K 2023-09-03 05:01:38,666 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~21.55 KB/22071, heapSize ~23.37 KB/23928, currentSize=3.08 KB/3153 for 1215e984d247204ad8c8b77a0af178ae in 26ms, sequenceid=36, compaction requested=false 2023-09-03 05:01:38,666 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1215e984d247204ad8c8b77a0af178ae: 2023-09-03 05:01:38,666 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=39.3 K, sizeToCheck=16.0 K 2023-09-03 05:01:38,666 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-03 05:01:38,667 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/131f2c5d082941cbbf3daca8507e2e7d because midkey is the same as first or last row 2023-09-03 05:01:40,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:40,651 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1215e984d247204ad8c8b77a0af178ae 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-03 05:01:40,686 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=1215e984d247204ad8c8b77a0af178ae, server=jenkins-hbase4.apache.org,35525,1693717287441 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-03 05:01:40,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] ipc.CallRunner(144): callId: 67 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:41714 deadline: 1693717310686, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=1215e984d247204ad8c8b77a0af178ae, server=jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:41,074 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=47 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/e5ab99b8368d49d385afb7aaad735009 2023-09-03 05:01:41,081 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/e5ab99b8368d49d385afb7aaad735009 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/e5ab99b8368d49d385afb7aaad735009 2023-09-03 05:01:41,086 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/e5ab99b8368d49d385afb7aaad735009, entries=8, sequenceid=47, filesize=13.0 K 2023-09-03 05:01:41,087 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 1215e984d247204ad8c8b77a0af178ae in 436ms, sequenceid=47, compaction requested=true 2023-09-03 05:01:41,087 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1215e984d247204ad8c8b77a0af178ae: 2023-09-03 05:01:41,087 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=52.3 K, sizeToCheck=16.0 K 2023-09-03 05:01:41,087 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-03 05:01:41,087 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/131f2c5d082941cbbf3daca8507e2e7d because midkey is the same as first or last row 2023-09-03 05:01:41,089 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:01:41,089 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-03 05:01:41,091 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 53524 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-03 05:01:41,091 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1912): 1215e984d247204ad8c8b77a0af178ae/info is initiating minor compaction (all files) 2023-09-03 05:01:41,091 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1215e984d247204ad8c8b77a0af178ae/info in TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. 2023-09-03 05:01:41,091 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/f0a5e10f7f464716b9e0d343bee1a67d, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/131f2c5d082941cbbf3daca8507e2e7d, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/e5ab99b8368d49d385afb7aaad735009] into tmpdir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp, totalSize=52.3 K 2023-09-03 05:01:41,091 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting f0a5e10f7f464716b9e0d343bee1a67d, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1693717298596 2023-09-03 05:01:41,092 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting 131f2c5d082941cbbf3daca8507e2e7d, keycount=21, bloomtype=ROW, size=26.3 K, encoding=NONE, compression=NONE, seqNum=36, earliestPutTs=1693717298608 2023-09-03 05:01:41,092 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting e5ab99b8368d49d385afb7aaad735009, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1693717298641 2023-09-03 05:01:41,104 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1215e984d247204ad8c8b77a0af178ae#info#compaction#18 average throughput is 18.54 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-03 05:01:41,125 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/be0e6d570c174fed8d2f27a6076663de as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/be0e6d570c174fed8d2f27a6076663de 2023-09-03 05:01:41,131 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1215e984d247204ad8c8b77a0af178ae/info of 1215e984d247204ad8c8b77a0af178ae into be0e6d570c174fed8d2f27a6076663de(size=42.9 K), total size for store is 42.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-03 05:01:41,131 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1215e984d247204ad8c8b77a0af178ae: 2023-09-03 05:01:41,131 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae., storeName=1215e984d247204ad8c8b77a0af178ae/info, priority=13, startTime=1693717301088; duration=0sec 2023-09-03 05:01:41,132 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=42.9 K, sizeToCheck=16.0 K 2023-09-03 05:01:41,132 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-03 05:01:41,133 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.StoreUtils(129): cannot split hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/be0e6d570c174fed8d2f27a6076663de because midkey is the same as first or last row 2023-09-03 05:01:41,133 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:01:50,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:50,722 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1215e984d247204ad8c8b77a0af178ae 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-03 05:01:50,735 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=73 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/e2531839050c4d9b813d95548e17cf56 2023-09-03 05:01:50,742 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/e2531839050c4d9b813d95548e17cf56 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/e2531839050c4d9b813d95548e17cf56 2023-09-03 05:01:50,747 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/e2531839050c4d9b813d95548e17cf56, entries=22, sequenceid=73, filesize=27.4 K 2023-09-03 05:01:50,748 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=5.13 KB/5255 for 1215e984d247204ad8c8b77a0af178ae in 26ms, sequenceid=73, compaction requested=false 2023-09-03 05:01:50,748 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1215e984d247204ad8c8b77a0af178ae: 2023-09-03 05:01:50,748 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=70.3 K, sizeToCheck=16.0 K 2023-09-03 05:01:50,748 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-03 05:01:50,748 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/be0e6d570c174fed8d2f27a6076663de because midkey is the same as first or last row 2023-09-03 05:01:52,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:52,733 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1215e984d247204ad8c8b77a0af178ae 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-03 05:01:52,749 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=84 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/59b8fc6b108c4aedad68c5b779d49247 2023-09-03 05:01:52,756 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/59b8fc6b108c4aedad68c5b779d49247 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/59b8fc6b108c4aedad68c5b779d49247 2023-09-03 05:01:52,762 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/59b8fc6b108c4aedad68c5b779d49247, entries=8, sequenceid=84, filesize=13.0 K 2023-09-03 05:01:52,763 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=20.53 KB/21020 for 1215e984d247204ad8c8b77a0af178ae in 30ms, sequenceid=84, compaction requested=true 2023-09-03 05:01:52,763 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1215e984d247204ad8c8b77a0af178ae: 2023-09-03 05:01:52,764 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=83.2 K, sizeToCheck=16.0 K 2023-09-03 05:01:52,764 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-03 05:01:52,764 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/be0e6d570c174fed8d2f27a6076663de because midkey is the same as first or last row 2023-09-03 05:01:52,764 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:01:52,764 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-03 05:01:52,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:52,764 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1215e984d247204ad8c8b77a0af178ae 1/1 column families, dataSize=21.55 KB heapSize=23.38 KB 2023-09-03 05:01:52,765 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 85243 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-03 05:01:52,765 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1912): 1215e984d247204ad8c8b77a0af178ae/info is initiating minor compaction (all files) 2023-09-03 05:01:52,765 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1215e984d247204ad8c8b77a0af178ae/info in TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. 2023-09-03 05:01:52,766 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/be0e6d570c174fed8d2f27a6076663de, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/e2531839050c4d9b813d95548e17cf56, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/59b8fc6b108c4aedad68c5b779d49247] into tmpdir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp, totalSize=83.2 K 2023-09-03 05:01:52,766 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting be0e6d570c174fed8d2f27a6076663de, keycount=37, bloomtype=ROW, size=42.9 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1693717298596 2023-09-03 05:01:52,767 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting e2531839050c4d9b813d95548e17cf56, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=73, earliestPutTs=1693717300652 2023-09-03 05:01:52,767 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting 59b8fc6b108c4aedad68c5b779d49247, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1693717310723 2023-09-03 05:01:52,789 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=21.55 KB at sequenceid=108 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/6dde0f01c7fb4e17ac8fec5d4a19500f 2023-09-03 05:01:52,793 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1215e984d247204ad8c8b77a0af178ae#info#compaction#22 average throughput is 16.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-03 05:01:52,796 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/6dde0f01c7fb4e17ac8fec5d4a19500f as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/6dde0f01c7fb4e17ac8fec5d4a19500f 2023-09-03 05:01:52,804 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/6dde0f01c7fb4e17ac8fec5d4a19500f, entries=21, sequenceid=108, filesize=26.3 K 2023-09-03 05:01:52,805 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~21.55 KB/22071, heapSize ~23.37 KB/23928, currentSize=8.21 KB/8408 for 1215e984d247204ad8c8b77a0af178ae in 41ms, sequenceid=108, compaction requested=false 2023-09-03 05:01:52,806 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1215e984d247204ad8c8b77a0af178ae: 2023-09-03 05:01:52,806 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=109.6 K, sizeToCheck=16.0 K 2023-09-03 05:01:52,806 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-03 05:01:52,806 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/be0e6d570c174fed8d2f27a6076663de because midkey is the same as first or last row 2023-09-03 05:01:52,815 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/ac9124e2340c4b16bde2e54c84cb7c92 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/ac9124e2340c4b16bde2e54c84cb7c92 2023-09-03 05:01:52,821 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1215e984d247204ad8c8b77a0af178ae/info of 1215e984d247204ad8c8b77a0af178ae into ac9124e2340c4b16bde2e54c84cb7c92(size=74.0 K), total size for store is 100.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-03 05:01:52,821 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1215e984d247204ad8c8b77a0af178ae: 2023-09-03 05:01:52,821 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae., storeName=1215e984d247204ad8c8b77a0af178ae/info, priority=13, startTime=1693717312764; duration=0sec 2023-09-03 05:01:52,821 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=100.3 K, sizeToCheck=16.0 K 2023-09-03 05:01:52,821 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-03 05:01:52,822 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit(227): Splitting TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae., compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:01:52,822 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:01:52,823 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39041] assignment.AssignmentManager(1140): Split request from jenkins-hbase4.apache.org,35525,1693717287441, parent={ENCODED => 1215e984d247204ad8c8b77a0af178ae, NAME => 'TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.', STARTKEY => '', ENDKEY => ''} splitKey=row0064 2023-09-03 05:01:52,830 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39041] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:52,836 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39041] procedure2.ProcedureExecutor(1028): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=1215e984d247204ad8c8b77a0af178ae, daughterA=84f7f1d3762c7969238da55751a158d9, daughterB=79d5aee206a8dfae817d860c826e2710 2023-09-03 05:01:52,837 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=1215e984d247204ad8c8b77a0af178ae, daughterA=84f7f1d3762c7969238da55751a158d9, daughterB=79d5aee206a8dfae817d860c826e2710 2023-09-03 05:01:52,837 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=1215e984d247204ad8c8b77a0af178ae, daughterA=84f7f1d3762c7969238da55751a158d9, daughterB=79d5aee206a8dfae817d860c826e2710 2023-09-03 05:01:52,837 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=1215e984d247204ad8c8b77a0af178ae, daughterA=84f7f1d3762c7969238da55751a158d9, daughterB=79d5aee206a8dfae817d860c826e2710 2023-09-03 05:01:52,844 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=1215e984d247204ad8c8b77a0af178ae, UNASSIGN}] 2023-09-03 05:01:52,846 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=1215e984d247204ad8c8b77a0af178ae, UNASSIGN 2023-09-03 05:01:52,846 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=1215e984d247204ad8c8b77a0af178ae, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:52,847 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693717312846"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693717312846"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693717312846"}]},"ts":"1693717312846"} 2023-09-03 05:01:52,848 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 1215e984d247204ad8c8b77a0af178ae, server=jenkins-hbase4.apache.org,35525,1693717287441}] 2023-09-03 05:01:53,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:53,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1215e984d247204ad8c8b77a0af178ae, disabling compactions & flushes 2023-09-03 05:01:53,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. 2023-09-03 05:01:53,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. 2023-09-03 05:01:53,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. after waiting 0 ms 2023-09-03 05:01:53,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. 2023-09-03 05:01:53,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1215e984d247204ad8c8b77a0af178ae 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-03 05:01:53,018 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=120 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/8dc68cf50e844f5a92a3519d2b3b9651 2023-09-03 05:01:53,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.tmp/info/8dc68cf50e844f5a92a3519d2b3b9651 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/8dc68cf50e844f5a92a3519d2b3b9651 2023-09-03 05:01:53,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/8dc68cf50e844f5a92a3519d2b3b9651, entries=8, sequenceid=120, filesize=13.0 K 2023-09-03 05:01:53,030 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=0 B/0 for 1215e984d247204ad8c8b77a0af178ae in 24ms, sequenceid=120, compaction requested=true 2023-09-03 05:01:53,036 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/f0a5e10f7f464716b9e0d343bee1a67d, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/131f2c5d082941cbbf3daca8507e2e7d, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/be0e6d570c174fed8d2f27a6076663de, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/e5ab99b8368d49d385afb7aaad735009, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/e2531839050c4d9b813d95548e17cf56, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/59b8fc6b108c4aedad68c5b779d49247] to archive 2023-09-03 05:01:53,037 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-09-03 05:01:53,039 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/f0a5e10f7f464716b9e0d343bee1a67d to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/f0a5e10f7f464716b9e0d343bee1a67d 2023-09-03 05:01:53,040 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/131f2c5d082941cbbf3daca8507e2e7d to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/131f2c5d082941cbbf3daca8507e2e7d 2023-09-03 05:01:53,042 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/be0e6d570c174fed8d2f27a6076663de to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/be0e6d570c174fed8d2f27a6076663de 2023-09-03 05:01:53,043 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/e5ab99b8368d49d385afb7aaad735009 to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/e5ab99b8368d49d385afb7aaad735009 2023-09-03 05:01:53,044 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/e2531839050c4d9b813d95548e17cf56 to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/e2531839050c4d9b813d95548e17cf56 2023-09-03 05:01:53,045 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/59b8fc6b108c4aedad68c5b779d49247 to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/59b8fc6b108c4aedad68c5b779d49247 2023-09-03 05:01:53,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/recovered.edits/123.seqid, newMaxSeqId=123, maxSeqId=1 2023-09-03 05:01:53,051 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. 2023-09-03 05:01:53,051 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1215e984d247204ad8c8b77a0af178ae: 2023-09-03 05:01:53,053 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:53,053 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=1215e984d247204ad8c8b77a0af178ae, regionState=CLOSED 2023-09-03 05:01:53,054 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693717313053"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693717313053"}]},"ts":"1693717313053"} 2023-09-03 05:01:53,057 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=14, resume processing ppid=13 2023-09-03 05:01:53,057 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 1215e984d247204ad8c8b77a0af178ae, server=jenkins-hbase4.apache.org,35525,1693717287441 in 207 msec 2023-09-03 05:01:53,059 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=13, resume processing ppid=12 2023-09-03 05:01:53,059 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=1215e984d247204ad8c8b77a0af178ae, UNASSIGN in 213 msec 2023-09-03 05:01:53,072 INFO [PEWorker-2] assignment.SplitTableRegionProcedure(694): pid=12 splitting 3 storefiles, region=1215e984d247204ad8c8b77a0af178ae, threads=3 2023-09-03 05:01:53,073 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/6dde0f01c7fb4e17ac8fec5d4a19500f for region: 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:53,073 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/8dc68cf50e844f5a92a3519d2b3b9651 for region: 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:53,073 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/ac9124e2340c4b16bde2e54c84cb7c92 for region: 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:53,083 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/8dc68cf50e844f5a92a3519d2b3b9651, top=true 2023-09-03 05:01:53,083 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/6dde0f01c7fb4e17ac8fec5d4a19500f, top=true 2023-09-03 05:01:53,089 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.splits/79d5aee206a8dfae817d860c826e2710/info/TestLogRolling-testLogRolling=1215e984d247204ad8c8b77a0af178ae-8dc68cf50e844f5a92a3519d2b3b9651 for child: 79d5aee206a8dfae817d860c826e2710, parent: 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:53,089 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/8dc68cf50e844f5a92a3519d2b3b9651 for region: 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:53,092 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/.splits/79d5aee206a8dfae817d860c826e2710/info/TestLogRolling-testLogRolling=1215e984d247204ad8c8b77a0af178ae-6dde0f01c7fb4e17ac8fec5d4a19500f for child: 79d5aee206a8dfae817d860c826e2710, parent: 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:53,092 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/6dde0f01c7fb4e17ac8fec5d4a19500f for region: 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:53,107 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/ac9124e2340c4b16bde2e54c84cb7c92 for region: 1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:01:53,107 DEBUG [PEWorker-2] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 1215e984d247204ad8c8b77a0af178ae Daughter A: 1 storefiles, Daughter B: 3 storefiles. 2023-09-03 05:01:53,133 DEBUG [PEWorker-2] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/84f7f1d3762c7969238da55751a158d9/recovered.edits/123.seqid, newMaxSeqId=123, maxSeqId=-1 2023-09-03 05:01:53,135 DEBUG [PEWorker-2] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/recovered.edits/123.seqid, newMaxSeqId=123, maxSeqId=-1 2023-09-03 05:01:53,137 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693717313137"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1693717313137"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1693717313137"}]},"ts":"1693717313137"} 2023-09-03 05:01:53,137 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693717313137"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693717313137"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693717313137"}]},"ts":"1693717313137"} 2023-09-03 05:01:53,137 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693717313137"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693717313137"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693717313137"}]},"ts":"1693717313137"} 2023-09-03 05:01:53,176 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35525] regionserver.HRegion(9158): Flush requested on 1588230740 2023-09-03 05:01:53,176 DEBUG [MemStoreFlusher.0] regionserver.FlushAllLargeStoresPolicy(69): Since none of the CFs were above the size, flushing all. 2023-09-03 05:01:53,176 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=4.82 KB heapSize=8.36 KB 2023-09-03 05:01:53,188 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=84f7f1d3762c7969238da55751a158d9, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=79d5aee206a8dfae817d860c826e2710, ASSIGN}] 2023-09-03 05:01:53,189 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=79d5aee206a8dfae817d860c826e2710, ASSIGN 2023-09-03 05:01:53,189 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=84f7f1d3762c7969238da55751a158d9, ASSIGN 2023-09-03 05:01:53,190 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=79d5aee206a8dfae817d860c826e2710, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,35525,1693717287441; forceNewPlan=false, retain=false 2023-09-03 05:01:53,190 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=84f7f1d3762c7969238da55751a158d9, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,35525,1693717287441; forceNewPlan=false, retain=false 2023-09-03 05:01:53,192 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.61 KB at sequenceid=17 (bloomFilter=false), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/.tmp/info/2ec05cbf07a1439ca109f5afbeab285c 2023-09-03 05:01:53,210 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=216 B at sequenceid=17 (bloomFilter=false), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/.tmp/table/e34e4df2b04f4c009bba4cbffcd289a6 2023-09-03 05:01:53,217 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/.tmp/info/2ec05cbf07a1439ca109f5afbeab285c as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/info/2ec05cbf07a1439ca109f5afbeab285c 2023-09-03 05:01:53,222 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/info/2ec05cbf07a1439ca109f5afbeab285c, entries=29, sequenceid=17, filesize=8.6 K 2023-09-03 05:01:53,224 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/.tmp/table/e34e4df2b04f4c009bba4cbffcd289a6 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/table/e34e4df2b04f4c009bba4cbffcd289a6 2023-09-03 05:01:53,230 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/table/e34e4df2b04f4c009bba4cbffcd289a6, entries=4, sequenceid=17, filesize=4.8 K 2023-09-03 05:01:53,231 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~4.82 KB/4934, heapSize ~8.08 KB/8272, currentSize=0 B/0 for 1588230740 in 55ms, sequenceid=17, compaction requested=false 2023-09-03 05:01:53,232 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-09-03 05:01:53,342 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=84f7f1d3762c7969238da55751a158d9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:53,342 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=79d5aee206a8dfae817d860c826e2710, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:53,342 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693717313341"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693717313341"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693717313341"}]},"ts":"1693717313341"} 2023-09-03 05:01:53,342 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693717313341"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693717313341"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693717313341"}]},"ts":"1693717313341"} 2023-09-03 05:01:53,344 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 84f7f1d3762c7969238da55751a158d9, server=jenkins-hbase4.apache.org,35525,1693717287441}] 2023-09-03 05:01:53,345 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 79d5aee206a8dfae817d860c826e2710, server=jenkins-hbase4.apache.org,35525,1693717287441}] 2023-09-03 05:01:53,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9. 2023-09-03 05:01:53,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 84f7f1d3762c7969238da55751a158d9, NAME => 'TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9.', STARTKEY => '', ENDKEY => 'row0064'} 2023-09-03 05:01:53,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling 84f7f1d3762c7969238da55751a158d9 2023-09-03 05:01:53,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:01:53,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 84f7f1d3762c7969238da55751a158d9 2023-09-03 05:01:53,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 84f7f1d3762c7969238da55751a158d9 2023-09-03 05:01:53,501 INFO [StoreOpener-84f7f1d3762c7969238da55751a158d9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 84f7f1d3762c7969238da55751a158d9 2023-09-03 05:01:53,502 DEBUG [StoreOpener-84f7f1d3762c7969238da55751a158d9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/84f7f1d3762c7969238da55751a158d9/info 2023-09-03 05:01:53,502 DEBUG [StoreOpener-84f7f1d3762c7969238da55751a158d9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/84f7f1d3762c7969238da55751a158d9/info 2023-09-03 05:01:53,503 INFO [StoreOpener-84f7f1d3762c7969238da55751a158d9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 84f7f1d3762c7969238da55751a158d9 columnFamilyName info 2023-09-03 05:01:53,516 DEBUG [StoreOpener-84f7f1d3762c7969238da55751a158d9-1] regionserver.HStore(539): loaded hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/84f7f1d3762c7969238da55751a158d9/info/ac9124e2340c4b16bde2e54c84cb7c92.1215e984d247204ad8c8b77a0af178ae->hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/ac9124e2340c4b16bde2e54c84cb7c92-bottom 2023-09-03 05:01:53,516 INFO [StoreOpener-84f7f1d3762c7969238da55751a158d9-1] regionserver.HStore(310): Store=84f7f1d3762c7969238da55751a158d9/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:01:53,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/84f7f1d3762c7969238da55751a158d9 2023-09-03 05:01:53,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/84f7f1d3762c7969238da55751a158d9 2023-09-03 05:01:53,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 84f7f1d3762c7969238da55751a158d9 2023-09-03 05:01:53,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 84f7f1d3762c7969238da55751a158d9; next sequenceid=124; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=764984, jitterRate=-0.027273163199424744}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-03 05:01:53,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 84f7f1d3762c7969238da55751a158d9: 2023-09-03 05:01:53,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9., pid=17, masterSystemTime=1693717313496 2023-09-03 05:01:53,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:01:53,525 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-09-03 05:01:53,526 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9. 2023-09-03 05:01:53,526 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1912): 84f7f1d3762c7969238da55751a158d9/info is initiating minor compaction (all files) 2023-09-03 05:01:53,526 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 84f7f1d3762c7969238da55751a158d9/info in TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9. 2023-09-03 05:01:53,526 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/84f7f1d3762c7969238da55751a158d9/info/ac9124e2340c4b16bde2e54c84cb7c92.1215e984d247204ad8c8b77a0af178ae->hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/ac9124e2340c4b16bde2e54c84cb7c92-bottom] into tmpdir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/84f7f1d3762c7969238da55751a158d9/.tmp, totalSize=74.0 K 2023-09-03 05:01:53,526 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting ac9124e2340c4b16bde2e54c84cb7c92.1215e984d247204ad8c8b77a0af178ae, keycount=33, bloomtype=ROW, size=74.0 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1693717298596 2023-09-03 05:01:53,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9. 2023-09-03 05:01:53,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9. 2023-09-03 05:01:53,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. 2023-09-03 05:01:53,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 79d5aee206a8dfae817d860c826e2710, NAME => 'TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.', STARTKEY => 'row0064', ENDKEY => ''} 2023-09-03 05:01:53,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:01:53,528 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=84f7f1d3762c7969238da55751a158d9, regionState=OPEN, openSeqNum=124, regionLocation=jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:53,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:01:53,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:01:53,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:01:53,528 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693717313528"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693717313528"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693717313528"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693717313528"}]},"ts":"1693717313528"} 2023-09-03 05:01:53,530 INFO [StoreOpener-79d5aee206a8dfae817d860c826e2710-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:01:53,531 DEBUG [StoreOpener-79d5aee206a8dfae817d860c826e2710-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info 2023-09-03 05:01:53,531 DEBUG [StoreOpener-79d5aee206a8dfae817d860c826e2710-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info 2023-09-03 05:01:53,531 INFO [StoreOpener-79d5aee206a8dfae817d860c826e2710-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 79d5aee206a8dfae817d860c826e2710 columnFamilyName info 2023-09-03 05:01:53,532 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=17, resume processing ppid=15 2023-09-03 05:01:53,533 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 84f7f1d3762c7969238da55751a158d9, server=jenkins-hbase4.apache.org,35525,1693717287441 in 186 msec 2023-09-03 05:01:53,535 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=84f7f1d3762c7969238da55751a158d9, ASSIGN in 345 msec 2023-09-03 05:01:53,538 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] throttle.PressureAwareThroughputController(145): 84f7f1d3762c7969238da55751a158d9#info#compaction#26 average throughput is 15.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-03 05:01:53,544 DEBUG [StoreOpener-79d5aee206a8dfae817d860c826e2710-1] regionserver.HStore(539): loaded hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/TestLogRolling-testLogRolling=1215e984d247204ad8c8b77a0af178ae-6dde0f01c7fb4e17ac8fec5d4a19500f 2023-09-03 05:01:53,555 DEBUG [StoreOpener-79d5aee206a8dfae817d860c826e2710-1] regionserver.HStore(539): loaded hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/TestLogRolling-testLogRolling=1215e984d247204ad8c8b77a0af178ae-8dc68cf50e844f5a92a3519d2b3b9651 2023-09-03 05:01:53,561 DEBUG [StoreOpener-79d5aee206a8dfae817d860c826e2710-1] regionserver.HStore(539): loaded hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/ac9124e2340c4b16bde2e54c84cb7c92.1215e984d247204ad8c8b77a0af178ae->hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/ac9124e2340c4b16bde2e54c84cb7c92-top 2023-09-03 05:01:53,561 INFO [StoreOpener-79d5aee206a8dfae817d860c826e2710-1] regionserver.HStore(310): Store=79d5aee206a8dfae817d860c826e2710/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:01:53,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710 2023-09-03 05:01:53,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710 2023-09-03 05:01:53,568 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:01:53,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 79d5aee206a8dfae817d860c826e2710; next sequenceid=124; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=840643, jitterRate=0.0689336508512497}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-03 05:01:53,570 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:01:53,570 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/84f7f1d3762c7969238da55751a158d9/.tmp/info/5bb59fc87cc149eca4eab947735f7352 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/84f7f1d3762c7969238da55751a158d9/info/5bb59fc87cc149eca4eab947735f7352 2023-09-03 05:01:53,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710., pid=18, masterSystemTime=1693717313496 2023-09-03 05:01:53,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:01:53,572 DEBUG [RS:0;jenkins-hbase4:35525-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-03 05:01:53,574 INFO [RS:0;jenkins-hbase4:35525-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. 2023-09-03 05:01:53,574 DEBUG [RS:0;jenkins-hbase4:35525-longCompactions-0] regionserver.HStore(1912): 79d5aee206a8dfae817d860c826e2710/info is initiating minor compaction (all files) 2023-09-03 05:01:53,575 INFO [RS:0;jenkins-hbase4:35525-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 79d5aee206a8dfae817d860c826e2710/info in TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. 2023-09-03 05:01:53,575 INFO [RS:0;jenkins-hbase4:35525-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/ac9124e2340c4b16bde2e54c84cb7c92.1215e984d247204ad8c8b77a0af178ae->hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/ac9124e2340c4b16bde2e54c84cb7c92-top, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/TestLogRolling-testLogRolling=1215e984d247204ad8c8b77a0af178ae-6dde0f01c7fb4e17ac8fec5d4a19500f, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/TestLogRolling-testLogRolling=1215e984d247204ad8c8b77a0af178ae-8dc68cf50e844f5a92a3519d2b3b9651] into tmpdir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp, totalSize=113.3 K 2023-09-03 05:01:53,575 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=79d5aee206a8dfae817d860c826e2710, regionState=OPEN, openSeqNum=124, regionLocation=jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:53,575 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693717313575"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693717313575"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693717313575"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693717313575"}]},"ts":"1693717313575"} 2023-09-03 05:01:53,575 DEBUG [RS:0;jenkins-hbase4:35525-longCompactions-0] compactions.Compactor(207): Compacting ac9124e2340c4b16bde2e54c84cb7c92.1215e984d247204ad8c8b77a0af178ae, keycount=33, bloomtype=ROW, size=74.0 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1693717298596 2023-09-03 05:01:53,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. 2023-09-03 05:01:53,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. 2023-09-03 05:01:53,577 DEBUG [RS:0;jenkins-hbase4:35525-longCompactions-0] compactions.Compactor(207): Compacting TestLogRolling-testLogRolling=1215e984d247204ad8c8b77a0af178ae-6dde0f01c7fb4e17ac8fec5d4a19500f, keycount=21, bloomtype=ROW, size=26.3 K, encoding=NONE, compression=NONE, seqNum=108, earliestPutTs=1693717312734 2023-09-03 05:01:53,578 DEBUG [RS:0;jenkins-hbase4:35525-longCompactions-0] compactions.Compactor(207): Compacting TestLogRolling-testLogRolling=1215e984d247204ad8c8b77a0af178ae-8dc68cf50e844f5a92a3519d2b3b9651, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=120, earliestPutTs=1693717312765 2023-09-03 05:01:53,581 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 1 (all) file(s) in 84f7f1d3762c7969238da55751a158d9/info of 84f7f1d3762c7969238da55751a158d9 into 5bb59fc87cc149eca4eab947735f7352(size=69.7 K), total size for store is 69.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-03 05:01:53,581 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 84f7f1d3762c7969238da55751a158d9: 2023-09-03 05:01:53,581 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=18, resume processing ppid=16 2023-09-03 05:01:53,581 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9., storeName=84f7f1d3762c7969238da55751a158d9/info, priority=15, startTime=1693717313524; duration=0sec 2023-09-03 05:01:53,581 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 79d5aee206a8dfae817d860c826e2710, server=jenkins-hbase4.apache.org,35525,1693717287441 in 233 msec 2023-09-03 05:01:53,581 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:01:53,584 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=16, resume processing ppid=12 2023-09-03 05:01:53,584 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=79d5aee206a8dfae817d860c826e2710, ASSIGN in 393 msec 2023-09-03 05:01:53,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=1215e984d247204ad8c8b77a0af178ae, daughterA=84f7f1d3762c7969238da55751a158d9, daughterB=79d5aee206a8dfae817d860c826e2710 in 754 msec 2023-09-03 05:01:53,590 INFO [RS:0;jenkins-hbase4:35525-longCompactions-0] throttle.PressureAwareThroughputController(145): 79d5aee206a8dfae817d860c826e2710#info#compaction#27 average throughput is 33.08 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-03 05:01:53,612 DEBUG [RS:0;jenkins-hbase4:35525-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/0e841b5a1d5649538a0f0f8d85bce347 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/0e841b5a1d5649538a0f0f8d85bce347 2023-09-03 05:01:53,618 INFO [RS:0;jenkins-hbase4:35525-longCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 79d5aee206a8dfae817d860c826e2710/info of 79d5aee206a8dfae817d860c826e2710 into 0e841b5a1d5649538a0f0f8d85bce347(size=39.0 K), total size for store is 39.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-03 05:01:53,619 DEBUG [RS:0;jenkins-hbase4:35525-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:01:53,619 INFO [RS:0;jenkins-hbase4:35525-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710., storeName=79d5aee206a8dfae817d860c826e2710/info, priority=13, startTime=1693717313571; duration=0sec 2023-09-03 05:01:53,619 DEBUG [RS:0;jenkins-hbase4:35525-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:01:54,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] ipc.CallRunner(144): callId: 107 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:41714 deadline: 1693717324775, exception=org.apache.hadoop.hbase.NotServingRegionException: TestLogRolling-testLogRolling,,1693717288580.1215e984d247204ad8c8b77a0af178ae. is not online on jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:01:58,596 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-03 05:02:04,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:02:04,810 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 79d5aee206a8dfae817d860c826e2710 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-03 05:02:04,823 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=135 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/b4528c6bf41d4fad949a0b92bdbf3b11 2023-09-03 05:02:04,829 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/b4528c6bf41d4fad949a0b92bdbf3b11 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/b4528c6bf41d4fad949a0b92bdbf3b11 2023-09-03 05:02:04,835 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/b4528c6bf41d4fad949a0b92bdbf3b11, entries=8, sequenceid=135, filesize=13.0 K 2023-09-03 05:02:04,836 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 79d5aee206a8dfae817d860c826e2710 in 26ms, sequenceid=135, compaction requested=false 2023-09-03 05:02:04,836 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:04,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:02:04,837 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 79d5aee206a8dfae817d860c826e2710 1/1 column families, dataSize=23.61 KB heapSize=25.59 KB 2023-09-03 05:02:04,848 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=23.61 KB at sequenceid=161 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/1cbe88497bb24ef396f5e8d22441f61d 2023-09-03 05:02:04,853 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/1cbe88497bb24ef396f5e8d22441f61d as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/1cbe88497bb24ef396f5e8d22441f61d 2023-09-03 05:02:04,862 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/1cbe88497bb24ef396f5e8d22441f61d, entries=23, sequenceid=161, filesize=28.4 K 2023-09-03 05:02:04,863 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~23.61 KB/24173, heapSize ~25.57 KB/26184, currentSize=1.03 KB/1051 for 79d5aee206a8dfae817d860c826e2710 in 26ms, sequenceid=161, compaction requested=true 2023-09-03 05:02:04,863 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:04,863 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:02:04,864 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-03 05:02:04,865 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 82277 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-03 05:02:04,865 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1912): 79d5aee206a8dfae817d860c826e2710/info is initiating minor compaction (all files) 2023-09-03 05:02:04,865 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 79d5aee206a8dfae817d860c826e2710/info in TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. 2023-09-03 05:02:04,865 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/0e841b5a1d5649538a0f0f8d85bce347, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/b4528c6bf41d4fad949a0b92bdbf3b11, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/1cbe88497bb24ef396f5e8d22441f61d] into tmpdir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp, totalSize=80.3 K 2023-09-03 05:02:04,865 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting 0e841b5a1d5649538a0f0f8d85bce347, keycount=33, bloomtype=ROW, size=39.0 K, encoding=NONE, compression=NONE, seqNum=120, earliestPutTs=1693717310728 2023-09-03 05:02:04,866 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting b4528c6bf41d4fad949a0b92bdbf3b11, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=135, earliestPutTs=1693717324802 2023-09-03 05:02:04,866 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting 1cbe88497bb24ef396f5e8d22441f61d, keycount=23, bloomtype=ROW, size=28.4 K, encoding=NONE, compression=NONE, seqNum=161, earliestPutTs=1693717324811 2023-09-03 05:02:04,881 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] throttle.PressureAwareThroughputController(145): 79d5aee206a8dfae817d860c826e2710#info#compaction#30 average throughput is 32.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-03 05:02:04,895 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/4a934a8ab5d4474a97b7e74bb0bcd17b as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/4a934a8ab5d4474a97b7e74bb0bcd17b 2023-09-03 05:02:04,901 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 79d5aee206a8dfae817d860c826e2710/info of 79d5aee206a8dfae817d860c826e2710 into 4a934a8ab5d4474a97b7e74bb0bcd17b(size=71.1 K), total size for store is 71.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-03 05:02:04,901 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:04,901 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710., storeName=79d5aee206a8dfae817d860c826e2710/info, priority=13, startTime=1693717324863; duration=0sec 2023-09-03 05:02:04,901 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:02:06,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:02:06,847 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 79d5aee206a8dfae817d860c826e2710 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-03 05:02:06,859 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=173 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/562f298a80e64833950f91fa28143308 2023-09-03 05:02:06,866 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/562f298a80e64833950f91fa28143308 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/562f298a80e64833950f91fa28143308 2023-09-03 05:02:06,871 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/562f298a80e64833950f91fa28143308, entries=8, sequenceid=173, filesize=13.0 K 2023-09-03 05:02:06,872 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 79d5aee206a8dfae817d860c826e2710 in 25ms, sequenceid=173, compaction requested=false 2023-09-03 05:02:06,872 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:06,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:02:06,873 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 79d5aee206a8dfae817d860c826e2710 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-03 05:02:06,887 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=198 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/fbcf2bcaf9634fcf9bb14b67342393c8 2023-09-03 05:02:06,893 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/fbcf2bcaf9634fcf9bb14b67342393c8 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/fbcf2bcaf9634fcf9bb14b67342393c8 2023-09-03 05:02:06,899 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/fbcf2bcaf9634fcf9bb14b67342393c8, entries=22, sequenceid=198, filesize=27.4 K 2023-09-03 05:02:06,900 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=3.08 KB/3153 for 79d5aee206a8dfae817d860c826e2710 in 27ms, sequenceid=198, compaction requested=true 2023-09-03 05:02:06,900 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:06,900 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:02:06,900 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-03 05:02:06,901 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 114110 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-03 05:02:06,902 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1912): 79d5aee206a8dfae817d860c826e2710/info is initiating minor compaction (all files) 2023-09-03 05:02:06,902 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 79d5aee206a8dfae817d860c826e2710/info in TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. 2023-09-03 05:02:06,902 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/4a934a8ab5d4474a97b7e74bb0bcd17b, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/562f298a80e64833950f91fa28143308, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/fbcf2bcaf9634fcf9bb14b67342393c8] into tmpdir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp, totalSize=111.4 K 2023-09-03 05:02:06,902 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting 4a934a8ab5d4474a97b7e74bb0bcd17b, keycount=64, bloomtype=ROW, size=71.1 K, encoding=NONE, compression=NONE, seqNum=161, earliestPutTs=1693717310728 2023-09-03 05:02:06,903 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting 562f298a80e64833950f91fa28143308, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=173, earliestPutTs=1693717324838 2023-09-03 05:02:06,903 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting fbcf2bcaf9634fcf9bb14b67342393c8, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=198, earliestPutTs=1693717326848 2023-09-03 05:02:06,917 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] throttle.PressureAwareThroughputController(145): 79d5aee206a8dfae817d860c826e2710#info#compaction#33 average throughput is 47.11 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-03 05:02:06,938 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/a0a7a6ad941f4ca39375de6a14a967d6 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/a0a7a6ad941f4ca39375de6a14a967d6 2023-09-03 05:02:06,945 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 79d5aee206a8dfae817d860c826e2710/info of 79d5aee206a8dfae817d860c826e2710 into a0a7a6ad941f4ca39375de6a14a967d6(size=102.0 K), total size for store is 102.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-03 05:02:06,945 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:06,945 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710., storeName=79d5aee206a8dfae817d860c826e2710/info, priority=13, startTime=1693717326900; duration=0sec 2023-09-03 05:02:06,945 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:02:08,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:02:08,883 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 79d5aee206a8dfae817d860c826e2710 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-03 05:02:08,897 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=210 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/47bfa225efc24939866c1dd0dd384381 2023-09-03 05:02:08,904 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/47bfa225efc24939866c1dd0dd384381 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/47bfa225efc24939866c1dd0dd384381 2023-09-03 05:02:08,909 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/47bfa225efc24939866c1dd0dd384381, entries=8, sequenceid=210, filesize=13.0 K 2023-09-03 05:02:08,910 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=20.53 KB/21020 for 79d5aee206a8dfae817d860c826e2710 in 27ms, sequenceid=210, compaction requested=false 2023-09-03 05:02:08,910 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:08,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:02:08,911 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 79d5aee206a8dfae817d860c826e2710 1/1 column families, dataSize=21.55 KB heapSize=23.38 KB 2023-09-03 05:02:08,921 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=21.55 KB at sequenceid=234 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/93a5e7e4f7f846839286b62049fef181 2023-09-03 05:02:08,926 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/93a5e7e4f7f846839286b62049fef181 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/93a5e7e4f7f846839286b62049fef181 2023-09-03 05:02:08,930 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/93a5e7e4f7f846839286b62049fef181, entries=21, sequenceid=234, filesize=26.4 K 2023-09-03 05:02:08,931 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~21.55 KB/22071, heapSize ~23.37 KB/23928, currentSize=6.16 KB/6306 for 79d5aee206a8dfae817d860c826e2710 in 20ms, sequenceid=234, compaction requested=true 2023-09-03 05:02:08,931 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:08,931 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:02:08,931 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-03 05:02:08,932 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 144753 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-03 05:02:08,932 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1912): 79d5aee206a8dfae817d860c826e2710/info is initiating minor compaction (all files) 2023-09-03 05:02:08,932 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 79d5aee206a8dfae817d860c826e2710/info in TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. 2023-09-03 05:02:08,932 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/a0a7a6ad941f4ca39375de6a14a967d6, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/47bfa225efc24939866c1dd0dd384381, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/93a5e7e4f7f846839286b62049fef181] into tmpdir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp, totalSize=141.4 K 2023-09-03 05:02:08,933 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting a0a7a6ad941f4ca39375de6a14a967d6, keycount=94, bloomtype=ROW, size=102.0 K, encoding=NONE, compression=NONE, seqNum=198, earliestPutTs=1693717310728 2023-09-03 05:02:08,933 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting 47bfa225efc24939866c1dd0dd384381, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=210, earliestPutTs=1693717326874 2023-09-03 05:02:08,933 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting 93a5e7e4f7f846839286b62049fef181, keycount=21, bloomtype=ROW, size=26.4 K, encoding=NONE, compression=NONE, seqNum=234, earliestPutTs=1693717328884 2023-09-03 05:02:08,943 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] throttle.PressureAwareThroughputController(145): 79d5aee206a8dfae817d860c826e2710#info#compaction#36 average throughput is 61.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-03 05:02:08,957 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/ab6b591bb812402f9c9364705ffdd5c9 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/ab6b591bb812402f9c9364705ffdd5c9 2023-09-03 05:02:08,962 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 79d5aee206a8dfae817d860c826e2710/info of 79d5aee206a8dfae817d860c826e2710 into ab6b591bb812402f9c9364705ffdd5c9(size=132.1 K), total size for store is 132.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-03 05:02:08,962 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:08,962 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710., storeName=79d5aee206a8dfae817d860c826e2710/info, priority=13, startTime=1693717328931; duration=0sec 2023-09-03 05:02:08,962 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:02:10,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:02:10,921 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 79d5aee206a8dfae817d860c826e2710 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-03 05:02:10,933 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=246 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/5cd00d29b9864699b33bb11c1370300b 2023-09-03 05:02:10,939 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/5cd00d29b9864699b33bb11c1370300b as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/5cd00d29b9864699b33bb11c1370300b 2023-09-03 05:02:10,945 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/5cd00d29b9864699b33bb11c1370300b, entries=8, sequenceid=246, filesize=13.0 K 2023-09-03 05:02:10,946 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 79d5aee206a8dfae817d860c826e2710 in 25ms, sequenceid=246, compaction requested=false 2023-09-03 05:02:10,946 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:10,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:02:10,946 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 79d5aee206a8dfae817d860c826e2710 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-03 05:02:10,955 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=79d5aee206a8dfae817d860c826e2710, server=jenkins-hbase4.apache.org,35525,1693717287441 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-03 05:02:10,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] ipc.CallRunner(144): callId: 238 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:41714 deadline: 1693717340955, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=79d5aee206a8dfae817d860c826e2710, server=jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:02:10,957 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=271 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/77794778023d436fbda917c91218f8cc 2023-09-03 05:02:10,962 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/77794778023d436fbda917c91218f8cc as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/77794778023d436fbda917c91218f8cc 2023-09-03 05:02:10,967 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/77794778023d436fbda917c91218f8cc, entries=22, sequenceid=271, filesize=27.4 K 2023-09-03 05:02:10,967 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=7.18 KB/7357 for 79d5aee206a8dfae817d860c826e2710 in 21ms, sequenceid=271, compaction requested=true 2023-09-03 05:02:10,967 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:10,967 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:02:10,967 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-03 05:02:10,969 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 176595 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-03 05:02:10,969 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1912): 79d5aee206a8dfae817d860c826e2710/info is initiating minor compaction (all files) 2023-09-03 05:02:10,969 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 79d5aee206a8dfae817d860c826e2710/info in TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. 2023-09-03 05:02:10,969 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/ab6b591bb812402f9c9364705ffdd5c9, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/5cd00d29b9864699b33bb11c1370300b, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/77794778023d436fbda917c91218f8cc] into tmpdir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp, totalSize=172.5 K 2023-09-03 05:02:10,969 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting ab6b591bb812402f9c9364705ffdd5c9, keycount=123, bloomtype=ROW, size=132.1 K, encoding=NONE, compression=NONE, seqNum=234, earliestPutTs=1693717310728 2023-09-03 05:02:10,970 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting 5cd00d29b9864699b33bb11c1370300b, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=246, earliestPutTs=1693717328911 2023-09-03 05:02:10,970 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting 77794778023d436fbda917c91218f8cc, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=271, earliestPutTs=1693717330922 2023-09-03 05:02:10,981 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] throttle.PressureAwareThroughputController(145): 79d5aee206a8dfae817d860c826e2710#info#compaction#39 average throughput is 51.12 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-03 05:02:10,993 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/73b5366905114670a3da4c27748e6d2f as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/73b5366905114670a3da4c27748e6d2f 2023-09-03 05:02:10,998 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 79d5aee206a8dfae817d860c826e2710/info of 79d5aee206a8dfae817d860c826e2710 into 73b5366905114670a3da4c27748e6d2f(size=163.1 K), total size for store is 163.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-03 05:02:10,998 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:10,998 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710., storeName=79d5aee206a8dfae817d860c826e2710/info, priority=13, startTime=1693717330967; duration=0sec 2023-09-03 05:02:10,999 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:02:21,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:02:21,053 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 79d5aee206a8dfae817d860c826e2710 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-03 05:02:21,064 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=283 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/b8b3777845ef4f269960b5f96b828182 2023-09-03 05:02:21,071 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/b8b3777845ef4f269960b5f96b828182 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/b8b3777845ef4f269960b5f96b828182 2023-09-03 05:02:21,076 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/b8b3777845ef4f269960b5f96b828182, entries=8, sequenceid=283, filesize=13.0 K 2023-09-03 05:02:21,076 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=0 B/0 for 79d5aee206a8dfae817d860c826e2710 in 23ms, sequenceid=283, compaction requested=false 2023-09-03 05:02:21,076 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:23,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:02:23,061 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 79d5aee206a8dfae817d860c826e2710 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-03 05:02:23,075 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=294 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/1cbb94a4694d40328cbec70ffa1fc2ba 2023-09-03 05:02:23,081 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/1cbb94a4694d40328cbec70ffa1fc2ba as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/1cbb94a4694d40328cbec70ffa1fc2ba 2023-09-03 05:02:23,087 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=79d5aee206a8dfae817d860c826e2710, server=jenkins-hbase4.apache.org,35525,1693717287441 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-03 05:02:23,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] ipc.CallRunner(144): callId: 270 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:41714 deadline: 1693717353087, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=79d5aee206a8dfae817d860c826e2710, server=jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:02:23,087 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/1cbb94a4694d40328cbec70ffa1fc2ba, entries=8, sequenceid=294, filesize=13.0 K 2023-09-03 05:02:23,088 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 79d5aee206a8dfae817d860c826e2710 in 27ms, sequenceid=294, compaction requested=true 2023-09-03 05:02:23,088 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:23,088 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:02:23,088 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-03 05:02:23,089 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 193617 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-03 05:02:23,089 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1912): 79d5aee206a8dfae817d860c826e2710/info is initiating minor compaction (all files) 2023-09-03 05:02:23,089 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 79d5aee206a8dfae817d860c826e2710/info in TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. 2023-09-03 05:02:23,089 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/73b5366905114670a3da4c27748e6d2f, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/b8b3777845ef4f269960b5f96b828182, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/1cbb94a4694d40328cbec70ffa1fc2ba] into tmpdir=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp, totalSize=189.1 K 2023-09-03 05:02:23,090 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting 73b5366905114670a3da4c27748e6d2f, keycount=153, bloomtype=ROW, size=163.1 K, encoding=NONE, compression=NONE, seqNum=271, earliestPutTs=1693717310728 2023-09-03 05:02:23,090 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting b8b3777845ef4f269960b5f96b828182, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=283, earliestPutTs=1693717330947 2023-09-03 05:02:23,090 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] compactions.Compactor(207): Compacting 1cbb94a4694d40328cbec70ffa1fc2ba, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=294, earliestPutTs=1693717343054 2023-09-03 05:02:23,103 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] throttle.PressureAwareThroughputController(145): 79d5aee206a8dfae817d860c826e2710#info#compaction#42 average throughput is 84.70 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-03 05:02:23,119 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/7770176acce74a29b877ac8e3da32bad as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/7770176acce74a29b877ac8e3da32bad 2023-09-03 05:02:23,125 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 79d5aee206a8dfae817d860c826e2710/info of 79d5aee206a8dfae817d860c826e2710 into 7770176acce74a29b877ac8e3da32bad(size=179.7 K), total size for store is 179.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-03 05:02:23,125 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:23,125 INFO [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710., storeName=79d5aee206a8dfae817d860c826e2710/info, priority=13, startTime=1693717343088; duration=0sec 2023-09-03 05:02:23,125 DEBUG [RS:0;jenkins-hbase4:35525-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-03 05:02:33,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35525] regionserver.HRegion(9158): Flush requested on 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:02:33,190 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 79d5aee206a8dfae817d860c826e2710 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-03 05:02:33,202 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=320 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/31afb75f675641fb8e3a560c26c409ac 2023-09-03 05:02:33,208 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/31afb75f675641fb8e3a560c26c409ac as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/31afb75f675641fb8e3a560c26c409ac 2023-09-03 05:02:33,214 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/31afb75f675641fb8e3a560c26c409ac, entries=22, sequenceid=320, filesize=27.4 K 2023-09-03 05:02:33,214 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=2.05 KB/2102 for 79d5aee206a8dfae817d860c826e2710 in 24ms, sequenceid=320, compaction requested=false 2023-09-03 05:02:33,215 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:35,194 INFO [Listener at localhost/45463] wal.AbstractTestLogRolling(188): after writing there are 0 log files 2023-09-03 05:02:35,210 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37087,DS-064a3f45-f12d-4222-996a-fd42de53fd59,DISK] 2023-09-03 05:02:35,211 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42837,DS-eae5636d-4bd8-4d07-9229-04293ad8a469,DISK] 2023-09-03 05:02:35,214 INFO [Listener at localhost/45463] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/WALs/jenkins-hbase4.apache.org,35525,1693717287441/jenkins-hbase4.apache.org%2C35525%2C1693717287441.1693717287821 with entries=308, filesize=300.35 KB; new WAL /user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/WALs/jenkins-hbase4.apache.org,35525,1693717287441/jenkins-hbase4.apache.org%2C35525%2C1693717287441.1693717355194 2023-09-03 05:02:35,214 DEBUG [Listener at localhost/45463] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37087,DS-064a3f45-f12d-4222-996a-fd42de53fd59,DISK], DatanodeInfoWithStorage[127.0.0.1:42837,DS-eae5636d-4bd8-4d07-9229-04293ad8a469,DISK]] 2023-09-03 05:02:35,214 DEBUG [Listener at localhost/45463] wal.AbstractFSWAL(716): hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/WALs/jenkins-hbase4.apache.org,35525,1693717287441/jenkins-hbase4.apache.org%2C35525%2C1693717287441.1693717287821 is not closed yet, will try archiving it next time 2023-09-03 05:02:35,219 INFO [Listener at localhost/45463] regionserver.HRegion(2745): Flushing 79d5aee206a8dfae817d860c826e2710 1/1 column families, dataSize=2.05 KB heapSize=2.45 KB 2023-09-03 05:02:35,232 INFO [Listener at localhost/45463] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.05 KB at sequenceid=325 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/ffa121f993a84a54a2f5cee5f932a2f5 2023-09-03 05:02:35,240 DEBUG [Listener at localhost/45463] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/.tmp/info/ffa121f993a84a54a2f5cee5f932a2f5 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/ffa121f993a84a54a2f5cee5f932a2f5 2023-09-03 05:02:35,244 INFO [Listener at localhost/45463] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/ffa121f993a84a54a2f5cee5f932a2f5, entries=2, sequenceid=325, filesize=6.8 K 2023-09-03 05:02:35,245 INFO [Listener at localhost/45463] regionserver.HRegion(2948): Finished flush of dataSize ~2.05 KB/2102, heapSize ~2.44 KB/2496, currentSize=0 B/0 for 79d5aee206a8dfae817d860c826e2710 in 26ms, sequenceid=325, compaction requested=true 2023-09-03 05:02:35,245 DEBUG [Listener at localhost/45463] regionserver.HRegion(2446): Flush status journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:35,245 DEBUG [Listener at localhost/45463] regionserver.HRegion(2446): Flush status journal for 84f7f1d3762c7969238da55751a158d9: 2023-09-03 05:02:35,245 INFO [Listener at localhost/45463] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.26 KB heapSize=4.19 KB 2023-09-03 05:02:35,257 INFO [Listener at localhost/45463] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.26 KB at sequenceid=24 (bloomFilter=false), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/.tmp/info/7a1a631e86654e7ab50fad4f6bd96e59 2023-09-03 05:02:35,263 DEBUG [Listener at localhost/45463] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/.tmp/info/7a1a631e86654e7ab50fad4f6bd96e59 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/info/7a1a631e86654e7ab50fad4f6bd96e59 2023-09-03 05:02:35,267 INFO [Listener at localhost/45463] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/info/7a1a631e86654e7ab50fad4f6bd96e59, entries=16, sequenceid=24, filesize=7.0 K 2023-09-03 05:02:35,268 INFO [Listener at localhost/45463] regionserver.HRegion(2948): Finished flush of dataSize ~2.26 KB/2312, heapSize ~3.67 KB/3760, currentSize=0 B/0 for 1588230740 in 23ms, sequenceid=24, compaction requested=false 2023-09-03 05:02:35,268 DEBUG [Listener at localhost/45463] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-09-03 05:02:35,269 INFO [Listener at localhost/45463] regionserver.HRegion(2745): Flushing d0ce8cdf1fe0a3b027b62dc1ee5530a6 1/1 column families, dataSize=78 B heapSize=488 B 2023-09-03 05:02:35,281 INFO [Listener at localhost/45463] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/namespace/d0ce8cdf1fe0a3b027b62dc1ee5530a6/.tmp/info/6369f5220f764646b22d75ab3c9b7880 2023-09-03 05:02:35,287 DEBUG [Listener at localhost/45463] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/namespace/d0ce8cdf1fe0a3b027b62dc1ee5530a6/.tmp/info/6369f5220f764646b22d75ab3c9b7880 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/namespace/d0ce8cdf1fe0a3b027b62dc1ee5530a6/info/6369f5220f764646b22d75ab3c9b7880 2023-09-03 05:02:35,292 INFO [Listener at localhost/45463] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/namespace/d0ce8cdf1fe0a3b027b62dc1ee5530a6/info/6369f5220f764646b22d75ab3c9b7880, entries=2, sequenceid=6, filesize=4.8 K 2023-09-03 05:02:35,292 INFO [Listener at localhost/45463] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for d0ce8cdf1fe0a3b027b62dc1ee5530a6 in 23ms, sequenceid=6, compaction requested=false 2023-09-03 05:02:35,293 DEBUG [Listener at localhost/45463] regionserver.HRegion(2446): Flush status journal for d0ce8cdf1fe0a3b027b62dc1ee5530a6: 2023-09-03 05:02:35,307 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42837,DS-eae5636d-4bd8-4d07-9229-04293ad8a469,DISK] 2023-09-03 05:02:35,307 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37087,DS-064a3f45-f12d-4222-996a-fd42de53fd59,DISK] 2023-09-03 05:02:35,310 INFO [Listener at localhost/45463] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/WALs/jenkins-hbase4.apache.org,35525,1693717287441/jenkins-hbase4.apache.org%2C35525%2C1693717287441.1693717355194 with entries=4, filesize=1.22 KB; new WAL /user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/WALs/jenkins-hbase4.apache.org,35525,1693717287441/jenkins-hbase4.apache.org%2C35525%2C1693717287441.1693717355293 2023-09-03 05:02:35,310 DEBUG [Listener at localhost/45463] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42837,DS-eae5636d-4bd8-4d07-9229-04293ad8a469,DISK], DatanodeInfoWithStorage[127.0.0.1:37087,DS-064a3f45-f12d-4222-996a-fd42de53fd59,DISK]] 2023-09-03 05:02:35,310 DEBUG [Listener at localhost/45463] wal.AbstractFSWAL(716): hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/WALs/jenkins-hbase4.apache.org,35525,1693717287441/jenkins-hbase4.apache.org%2C35525%2C1693717287441.1693717355194 is not closed yet, will try archiving it next time 2023-09-03 05:02:35,311 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/WALs/jenkins-hbase4.apache.org,35525,1693717287441/jenkins-hbase4.apache.org%2C35525%2C1693717287441.1693717287821 to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/oldWALs/jenkins-hbase4.apache.org%2C35525%2C1693717287441.1693717287821 2023-09-03 05:02:35,311 INFO [Listener at localhost/45463] hbase.Waiter(180): Waiting up to [5,000] milli-secs(wait.for.ratio=[1]) 2023-09-03 05:02:35,313 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/WALs/jenkins-hbase4.apache.org,35525,1693717287441/jenkins-hbase4.apache.org%2C35525%2C1693717287441.1693717355194 to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/oldWALs/jenkins-hbase4.apache.org%2C35525%2C1693717287441.1693717355194 2023-09-03 05:02:35,412 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-09-03 05:02:35,412 INFO [Listener at localhost/45463] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-09-03 05:02:35,412 DEBUG [Listener at localhost/45463] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x26cadc1d to 127.0.0.1:53078 2023-09-03 05:02:35,412 DEBUG [Listener at localhost/45463] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:02:35,412 DEBUG [Listener at localhost/45463] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-09-03 05:02:35,412 DEBUG [Listener at localhost/45463] util.JVMClusterUtil(257): Found active master hash=235752828, stopped=false 2023-09-03 05:02:35,412 INFO [Listener at localhost/45463] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,39041,1693717287389 2023-09-03 05:02:35,414 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-03 05:02:35,414 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): regionserver:35525-0x100702a09720001, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-03 05:02:35,414 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:02:35,414 INFO [Listener at localhost/45463] procedure2.ProcedureExecutor(628): Stopping 2023-09-03 05:02:35,414 DEBUG [Listener at localhost/45463] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1769a049 to 127.0.0.1:53078 2023-09-03 05:02:35,415 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:02:35,415 DEBUG [Listener at localhost/45463] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:02:35,415 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35525-0x100702a09720001, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:02:35,415 INFO [Listener at localhost/45463] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,35525,1693717287441' ***** 2023-09-03 05:02:35,415 INFO [Listener at localhost/45463] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-09-03 05:02:35,415 INFO [RS:0;jenkins-hbase4:35525] regionserver.HeapMemoryManager(220): Stopping 2023-09-03 05:02:35,415 INFO [RS:0;jenkins-hbase4:35525] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-09-03 05:02:35,415 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-09-03 05:02:35,415 INFO [RS:0;jenkins-hbase4:35525] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-09-03 05:02:35,416 INFO [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(3305): Received CLOSE for 79d5aee206a8dfae817d860c826e2710 2023-09-03 05:02:35,416 INFO [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(3305): Received CLOSE for 84f7f1d3762c7969238da55751a158d9 2023-09-03 05:02:35,416 INFO [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(3305): Received CLOSE for d0ce8cdf1fe0a3b027b62dc1ee5530a6 2023-09-03 05:02:35,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 79d5aee206a8dfae817d860c826e2710, disabling compactions & flushes 2023-09-03 05:02:35,416 INFO [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:02:35,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. 2023-09-03 05:02:35,416 DEBUG [RS:0;jenkins-hbase4:35525] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x42e5ad83 to 127.0.0.1:53078 2023-09-03 05:02:35,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. 2023-09-03 05:02:35,416 DEBUG [RS:0;jenkins-hbase4:35525] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:02:35,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. after waiting 0 ms 2023-09-03 05:02:35,416 INFO [RS:0;jenkins-hbase4:35525] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-09-03 05:02:35,416 INFO [RS:0;jenkins-hbase4:35525] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-09-03 05:02:35,416 INFO [RS:0;jenkins-hbase4:35525] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-09-03 05:02:35,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. 2023-09-03 05:02:35,416 INFO [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-09-03 05:02:35,416 INFO [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-09-03 05:02:35,417 DEBUG [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(1478): Online Regions={79d5aee206a8dfae817d860c826e2710=TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710., 84f7f1d3762c7969238da55751a158d9=TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9., 1588230740=hbase:meta,,1.1588230740, d0ce8cdf1fe0a3b027b62dc1ee5530a6=hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6.} 2023-09-03 05:02:35,418 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-03 05:02:35,418 DEBUG [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(1504): Waiting on 1588230740, 79d5aee206a8dfae817d860c826e2710, 84f7f1d3762c7969238da55751a158d9, d0ce8cdf1fe0a3b027b62dc1ee5530a6 2023-09-03 05:02:35,419 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-03 05:02:35,420 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-03 05:02:35,421 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-03 05:02:35,422 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-03 05:02:35,433 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/ac9124e2340c4b16bde2e54c84cb7c92.1215e984d247204ad8c8b77a0af178ae->hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/ac9124e2340c4b16bde2e54c84cb7c92-top, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/TestLogRolling-testLogRolling=1215e984d247204ad8c8b77a0af178ae-6dde0f01c7fb4e17ac8fec5d4a19500f, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/0e841b5a1d5649538a0f0f8d85bce347, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/TestLogRolling-testLogRolling=1215e984d247204ad8c8b77a0af178ae-8dc68cf50e844f5a92a3519d2b3b9651, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/b4528c6bf41d4fad949a0b92bdbf3b11, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/4a934a8ab5d4474a97b7e74bb0bcd17b, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/1cbe88497bb24ef396f5e8d22441f61d, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/562f298a80e64833950f91fa28143308, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/a0a7a6ad941f4ca39375de6a14a967d6, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/fbcf2bcaf9634fcf9bb14b67342393c8, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/47bfa225efc24939866c1dd0dd384381, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/ab6b591bb812402f9c9364705ffdd5c9, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/93a5e7e4f7f846839286b62049fef181, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/5cd00d29b9864699b33bb11c1370300b, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/73b5366905114670a3da4c27748e6d2f, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/77794778023d436fbda917c91218f8cc, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/b8b3777845ef4f269960b5f96b828182, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/1cbb94a4694d40328cbec70ffa1fc2ba] to archive 2023-09-03 05:02:35,434 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-09-03 05:02:35,436 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/ac9124e2340c4b16bde2e54c84cb7c92.1215e984d247204ad8c8b77a0af178ae to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/ac9124e2340c4b16bde2e54c84cb7c92.1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:02:35,436 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/meta/1588230740/recovered.edits/27.seqid, newMaxSeqId=27, maxSeqId=1 2023-09-03 05:02:35,437 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-03 05:02:35,437 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-03 05:02:35,438 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-03 05:02:35,438 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-09-03 05:02:35,438 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/TestLogRolling-testLogRolling=1215e984d247204ad8c8b77a0af178ae-6dde0f01c7fb4e17ac8fec5d4a19500f to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/TestLogRolling-testLogRolling=1215e984d247204ad8c8b77a0af178ae-6dde0f01c7fb4e17ac8fec5d4a19500f 2023-09-03 05:02:35,440 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/0e841b5a1d5649538a0f0f8d85bce347 to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/0e841b5a1d5649538a0f0f8d85bce347 2023-09-03 05:02:35,441 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/TestLogRolling-testLogRolling=1215e984d247204ad8c8b77a0af178ae-8dc68cf50e844f5a92a3519d2b3b9651 to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/TestLogRolling-testLogRolling=1215e984d247204ad8c8b77a0af178ae-8dc68cf50e844f5a92a3519d2b3b9651 2023-09-03 05:02:35,442 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/b4528c6bf41d4fad949a0b92bdbf3b11 to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/b4528c6bf41d4fad949a0b92bdbf3b11 2023-09-03 05:02:35,443 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/4a934a8ab5d4474a97b7e74bb0bcd17b to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/4a934a8ab5d4474a97b7e74bb0bcd17b 2023-09-03 05:02:35,444 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/1cbe88497bb24ef396f5e8d22441f61d to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/1cbe88497bb24ef396f5e8d22441f61d 2023-09-03 05:02:35,445 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/562f298a80e64833950f91fa28143308 to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/562f298a80e64833950f91fa28143308 2023-09-03 05:02:35,446 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/a0a7a6ad941f4ca39375de6a14a967d6 to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/a0a7a6ad941f4ca39375de6a14a967d6 2023-09-03 05:02:35,448 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/fbcf2bcaf9634fcf9bb14b67342393c8 to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/fbcf2bcaf9634fcf9bb14b67342393c8 2023-09-03 05:02:35,449 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/47bfa225efc24939866c1dd0dd384381 to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/47bfa225efc24939866c1dd0dd384381 2023-09-03 05:02:35,450 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/ab6b591bb812402f9c9364705ffdd5c9 to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/ab6b591bb812402f9c9364705ffdd5c9 2023-09-03 05:02:35,451 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/93a5e7e4f7f846839286b62049fef181 to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/93a5e7e4f7f846839286b62049fef181 2023-09-03 05:02:35,452 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/5cd00d29b9864699b33bb11c1370300b to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/5cd00d29b9864699b33bb11c1370300b 2023-09-03 05:02:35,454 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/73b5366905114670a3da4c27748e6d2f to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/73b5366905114670a3da4c27748e6d2f 2023-09-03 05:02:35,455 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/77794778023d436fbda917c91218f8cc to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/77794778023d436fbda917c91218f8cc 2023-09-03 05:02:35,456 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/b8b3777845ef4f269960b5f96b828182 to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/b8b3777845ef4f269960b5f96b828182 2023-09-03 05:02:35,457 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/1cbb94a4694d40328cbec70ffa1fc2ba to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/info/1cbb94a4694d40328cbec70ffa1fc2ba 2023-09-03 05:02:35,461 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/79d5aee206a8dfae817d860c826e2710/recovered.edits/328.seqid, newMaxSeqId=328, maxSeqId=123 2023-09-03 05:02:35,462 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. 2023-09-03 05:02:35,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 79d5aee206a8dfae817d860c826e2710: 2023-09-03 05:02:35,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,row0064,1693717312830.79d5aee206a8dfae817d860c826e2710. 2023-09-03 05:02:35,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 84f7f1d3762c7969238da55751a158d9, disabling compactions & flushes 2023-09-03 05:02:35,462 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9. 2023-09-03 05:02:35,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9. 2023-09-03 05:02:35,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9. after waiting 0 ms 2023-09-03 05:02:35,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9. 2023-09-03 05:02:35,463 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/84f7f1d3762c7969238da55751a158d9/info/ac9124e2340c4b16bde2e54c84cb7c92.1215e984d247204ad8c8b77a0af178ae->hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/1215e984d247204ad8c8b77a0af178ae/info/ac9124e2340c4b16bde2e54c84cb7c92-bottom] to archive 2023-09-03 05:02:35,464 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-09-03 05:02:35,465 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/84f7f1d3762c7969238da55751a158d9/info/ac9124e2340c4b16bde2e54c84cb7c92.1215e984d247204ad8c8b77a0af178ae to hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/archive/data/default/TestLogRolling-testLogRolling/84f7f1d3762c7969238da55751a158d9/info/ac9124e2340c4b16bde2e54c84cb7c92.1215e984d247204ad8c8b77a0af178ae 2023-09-03 05:02:35,469 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/default/TestLogRolling-testLogRolling/84f7f1d3762c7969238da55751a158d9/recovered.edits/128.seqid, newMaxSeqId=128, maxSeqId=123 2023-09-03 05:02:35,470 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9. 2023-09-03 05:02:35,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 84f7f1d3762c7969238da55751a158d9: 2023-09-03 05:02:35,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,,1693717312830.84f7f1d3762c7969238da55751a158d9. 2023-09-03 05:02:35,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d0ce8cdf1fe0a3b027b62dc1ee5530a6, disabling compactions & flushes 2023-09-03 05:02:35,470 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6. 2023-09-03 05:02:35,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6. 2023-09-03 05:02:35,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6. after waiting 0 ms 2023-09-03 05:02:35,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6. 2023-09-03 05:02:35,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/data/hbase/namespace/d0ce8cdf1fe0a3b027b62dc1ee5530a6/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-09-03 05:02:35,474 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6. 2023-09-03 05:02:35,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d0ce8cdf1fe0a3b027b62dc1ee5530a6: 2023-09-03 05:02:35,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693717288013.d0ce8cdf1fe0a3b027b62dc1ee5530a6. 2023-09-03 05:02:35,620 INFO [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,35525,1693717287441; all regions closed. 2023-09-03 05:02:35,626 DEBUG [RS:0;jenkins-hbase4:35525] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/oldWALs 2023-09-03 05:02:35,626 INFO [RS:0;jenkins-hbase4:35525] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35525%2C1693717287441.meta:.meta(num 1693717287953) 2023-09-03 05:02:35,630 DEBUG [RS:0;jenkins-hbase4:35525] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/oldWALs 2023-09-03 05:02:35,630 INFO [RS:0;jenkins-hbase4:35525] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35525%2C1693717287441:(num 1693717355293) 2023-09-03 05:02:35,630 DEBUG [RS:0;jenkins-hbase4:35525] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:02:35,630 INFO [RS:0;jenkins-hbase4:35525] regionserver.LeaseManager(133): Closed leases 2023-09-03 05:02:35,631 INFO [RS:0;jenkins-hbase4:35525] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-09-03 05:02:35,631 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-03 05:02:35,631 INFO [RS:0;jenkins-hbase4:35525] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35525 2023-09-03 05:02:35,635 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): regionserver:35525-0x100702a09720001, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,35525,1693717287441 2023-09-03 05:02:35,635 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-03 05:02:35,635 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): regionserver:35525-0x100702a09720001, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-03 05:02:35,636 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,35525,1693717287441] 2023-09-03 05:02:35,636 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,35525,1693717287441; numProcessing=1 2023-09-03 05:02:35,637 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,35525,1693717287441 already deleted, retry=false 2023-09-03 05:02:35,637 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,35525,1693717287441 expired; onlineServers=0 2023-09-03 05:02:35,637 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39041,1693717287389' ***** 2023-09-03 05:02:35,638 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-09-03 05:02:35,638 DEBUG [M:0;jenkins-hbase4:39041] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@a32c865, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-03 05:02:35,638 INFO [M:0;jenkins-hbase4:39041] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39041,1693717287389 2023-09-03 05:02:35,638 INFO [M:0;jenkins-hbase4:39041] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39041,1693717287389; all regions closed. 2023-09-03 05:02:35,638 DEBUG [M:0;jenkins-hbase4:39041] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:02:35,638 DEBUG [M:0;jenkins-hbase4:39041] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-09-03 05:02:35,638 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-09-03 05:02:35,638 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693717287596] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693717287596,5,FailOnTimeoutGroup] 2023-09-03 05:02:35,638 DEBUG [M:0;jenkins-hbase4:39041] cleaner.HFileCleaner(317): Stopping file delete threads 2023-09-03 05:02:35,638 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693717287596] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693717287596,5,FailOnTimeoutGroup] 2023-09-03 05:02:35,639 INFO [M:0;jenkins-hbase4:39041] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-09-03 05:02:35,639 INFO [M:0;jenkins-hbase4:39041] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-09-03 05:02:35,639 INFO [M:0;jenkins-hbase4:39041] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-09-03 05:02:35,639 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-09-03 05:02:35,639 DEBUG [M:0;jenkins-hbase4:39041] master.HMaster(1512): Stopping service threads 2023-09-03 05:02:35,640 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:02:35,640 INFO [M:0;jenkins-hbase4:39041] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-09-03 05:02:35,640 ERROR [M:0;jenkins-hbase4:39041] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-09-03 05:02:35,640 INFO [M:0;jenkins-hbase4:39041] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-09-03 05:02:35,640 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-09-03 05:02:35,640 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-03 05:02:35,640 DEBUG [M:0;jenkins-hbase4:39041] zookeeper.ZKUtil(398): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-09-03 05:02:35,640 WARN [M:0;jenkins-hbase4:39041] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-09-03 05:02:35,640 INFO [M:0;jenkins-hbase4:39041] assignment.AssignmentManager(315): Stopping assignment manager 2023-09-03 05:02:35,641 INFO [M:0;jenkins-hbase4:39041] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-09-03 05:02:35,641 DEBUG [M:0;jenkins-hbase4:39041] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-03 05:02:35,641 INFO [M:0;jenkins-hbase4:39041] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:02:35,641 DEBUG [M:0;jenkins-hbase4:39041] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:02:35,641 DEBUG [M:0;jenkins-hbase4:39041] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-03 05:02:35,641 DEBUG [M:0;jenkins-hbase4:39041] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:02:35,641 INFO [M:0;jenkins-hbase4:39041] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=64.71 KB heapSize=78.42 KB 2023-09-03 05:02:35,654 INFO [M:0;jenkins-hbase4:39041] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=64.71 KB at sequenceid=160 (bloomFilter=true), to=hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/2892801f7961429599b45245b9c57931 2023-09-03 05:02:35,660 INFO [M:0;jenkins-hbase4:39041] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2892801f7961429599b45245b9c57931 2023-09-03 05:02:35,660 DEBUG [M:0;jenkins-hbase4:39041] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/2892801f7961429599b45245b9c57931 as hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2892801f7961429599b45245b9c57931 2023-09-03 05:02:35,665 INFO [M:0;jenkins-hbase4:39041] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2892801f7961429599b45245b9c57931 2023-09-03 05:02:35,665 INFO [M:0;jenkins-hbase4:39041] regionserver.HStore(1080): Added hdfs://localhost:39747/user/jenkins/test-data/15c23ba3-50b3-3b31-8190-b057b53e5a3d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2892801f7961429599b45245b9c57931, entries=18, sequenceid=160, filesize=6.9 K 2023-09-03 05:02:35,666 INFO [M:0;jenkins-hbase4:39041] regionserver.HRegion(2948): Finished flush of dataSize ~64.71 KB/66268, heapSize ~78.41 KB/80288, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 25ms, sequenceid=160, compaction requested=false 2023-09-03 05:02:35,667 INFO [M:0;jenkins-hbase4:39041] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:02:35,667 DEBUG [M:0;jenkins-hbase4:39041] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-03 05:02:35,670 INFO [M:0;jenkins-hbase4:39041] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-09-03 05:02:35,670 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-03 05:02:35,671 INFO [M:0;jenkins-hbase4:39041] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39041 2023-09-03 05:02:35,674 DEBUG [M:0;jenkins-hbase4:39041] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,39041,1693717287389 already deleted, retry=false 2023-09-03 05:02:35,698 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-09-03 05:02:35,737 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): regionserver:35525-0x100702a09720001, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:02:35,737 INFO [RS:0;jenkins-hbase4:35525] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,35525,1693717287441; zookeeper connection closed. 2023-09-03 05:02:35,737 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): regionserver:35525-0x100702a09720001, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:02:35,737 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5ad7b0ef] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5ad7b0ef 2023-09-03 05:02:35,738 INFO [Listener at localhost/45463] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-09-03 05:02:35,837 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:02:35,837 INFO [M:0;jenkins-hbase4:39041] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39041,1693717287389; zookeeper connection closed. 2023-09-03 05:02:35,837 DEBUG [Listener at localhost/45463-EventThread] zookeeper.ZKWatcher(600): master:39041-0x100702a09720000, quorum=127.0.0.1:53078, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:02:35,838 WARN [Listener at localhost/45463] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-03 05:02:35,841 INFO [Listener at localhost/45463] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:02:35,946 WARN [BP-1922553724-172.31.14.131-1693717286848 heartbeating to localhost/127.0.0.1:39747] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-03 05:02:35,946 WARN [BP-1922553724-172.31.14.131-1693717286848 heartbeating to localhost/127.0.0.1:39747] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1922553724-172.31.14.131-1693717286848 (Datanode Uuid e27a9e9b-c163-4ef4-bc88-e927d7f3b165) service to localhost/127.0.0.1:39747 2023-09-03 05:02:35,946 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/cluster_570609d1-7cf6-2009-b5e0-6d4de4b161ae/dfs/data/data3/current/BP-1922553724-172.31.14.131-1693717286848] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:02:35,947 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/cluster_570609d1-7cf6-2009-b5e0-6d4de4b161ae/dfs/data/data4/current/BP-1922553724-172.31.14.131-1693717286848] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:02:35,948 WARN [Listener at localhost/45463] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-03 05:02:35,951 INFO [Listener at localhost/45463] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:02:36,055 WARN [BP-1922553724-172.31.14.131-1693717286848 heartbeating to localhost/127.0.0.1:39747] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-03 05:02:36,055 WARN [BP-1922553724-172.31.14.131-1693717286848 heartbeating to localhost/127.0.0.1:39747] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1922553724-172.31.14.131-1693717286848 (Datanode Uuid ac4a592c-517b-4ef9-9ce4-16f8ab144043) service to localhost/127.0.0.1:39747 2023-09-03 05:02:36,056 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/cluster_570609d1-7cf6-2009-b5e0-6d4de4b161ae/dfs/data/data1/current/BP-1922553724-172.31.14.131-1693717286848] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:02:36,057 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/cluster_570609d1-7cf6-2009-b5e0-6d4de4b161ae/dfs/data/data2/current/BP-1922553724-172.31.14.131-1693717286848] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:02:36,079 INFO [Listener at localhost/45463] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:02:36,196 INFO [Listener at localhost/45463] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-09-03 05:02:36,235 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-09-03 05:02:36,245 INFO [Listener at localhost/45463] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=75 (was 64) - Thread LEAK? -, OpenFileDescriptor=464 (was 433) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=34 (was 93), ProcessCount=167 (was 167), AvailableMemoryMB=8299 (was 8356) 2023-09-03 05:02:36,253 INFO [Listener at localhost/45463] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=76, OpenFileDescriptor=464, MaxFileDescriptor=60000, SystemLoadAverage=34, ProcessCount=167, AvailableMemoryMB=8299 2023-09-03 05:02:36,253 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-09-03 05:02:36,253 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/hadoop.log.dir so I do NOT create it in target/test-data/428ec61a-a342-5973-b791-b3100057782d 2023-09-03 05:02:36,253 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/896df6f1-906b-f42a-a479-324b101022d7/hadoop.tmp.dir so I do NOT create it in target/test-data/428ec61a-a342-5973-b791-b3100057782d 2023-09-03 05:02:36,253 INFO [Listener at localhost/45463] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/cluster_858f4b01-d619-9969-c868-5f1e13a4095d, deleteOnExit=true 2023-09-03 05:02:36,254 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-09-03 05:02:36,254 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/test.cache.data in system properties and HBase conf 2023-09-03 05:02:36,254 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/hadoop.tmp.dir in system properties and HBase conf 2023-09-03 05:02:36,254 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/hadoop.log.dir in system properties and HBase conf 2023-09-03 05:02:36,254 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/mapreduce.cluster.local.dir in system properties and HBase conf 2023-09-03 05:02:36,254 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-09-03 05:02:36,254 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-09-03 05:02:36,254 DEBUG [Listener at localhost/45463] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-09-03 05:02:36,254 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-09-03 05:02:36,255 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-09-03 05:02:36,255 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-09-03 05:02:36,255 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-03 05:02:36,255 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-09-03 05:02:36,255 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-09-03 05:02:36,255 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-03 05:02:36,255 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-03 05:02:36,255 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-09-03 05:02:36,255 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/nfs.dump.dir in system properties and HBase conf 2023-09-03 05:02:36,255 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/java.io.tmpdir in system properties and HBase conf 2023-09-03 05:02:36,255 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-03 05:02:36,256 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-09-03 05:02:36,256 INFO [Listener at localhost/45463] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-09-03 05:02:36,261 WARN [Listener at localhost/45463] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-03 05:02:36,261 WARN [Listener at localhost/45463] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-03 05:02:36,306 WARN [Listener at localhost/45463] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 05:02:36,307 INFO [Listener at localhost/45463] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 05:02:36,312 INFO [Listener at localhost/45463] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/java.io.tmpdir/Jetty_localhost_34121_hdfs____.85x0dj/webapp 2023-09-03 05:02:36,403 INFO [Listener at localhost/45463] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34121 2023-09-03 05:02:36,410 WARN [Listener at localhost/45463] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-03 05:02:36,410 WARN [Listener at localhost/45463] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-03 05:02:36,459 WARN [Listener at localhost/34487] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 05:02:36,471 WARN [Listener at localhost/34487] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-03 05:02:36,474 WARN [Listener at localhost/34487] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 05:02:36,475 INFO [Listener at localhost/34487] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 05:02:36,522 INFO [Listener at localhost/34487] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/java.io.tmpdir/Jetty_localhost_46419_datanode____a6cfi9/webapp 2023-09-03 05:02:36,616 INFO [Listener at localhost/34487] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46419 2023-09-03 05:02:36,623 WARN [Listener at localhost/41705] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 05:02:36,636 WARN [Listener at localhost/41705] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-03 05:02:36,638 WARN [Listener at localhost/41705] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-03 05:02:36,640 INFO [Listener at localhost/41705] log.Slf4jLog(67): jetty-6.1.26 2023-09-03 05:02:36,644 INFO [Listener at localhost/41705] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/java.io.tmpdir/Jetty_localhost_33589_datanode____.kgx75v/webapp 2023-09-03 05:02:36,721 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x9fcaa4ede383d3e0: Processing first storage report for DS-4e88db21-5d8e-4fc0-a21a-5e7953c5711c from datanode ce93aba6-c719-4530-a1d7-3d6aa1cc0935 2023-09-03 05:02:36,721 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x9fcaa4ede383d3e0: from storage DS-4e88db21-5d8e-4fc0-a21a-5e7953c5711c node DatanodeRegistration(127.0.0.1:38701, datanodeUuid=ce93aba6-c719-4530-a1d7-3d6aa1cc0935, infoPort=37119, infoSecurePort=0, ipcPort=41705, storageInfo=lv=-57;cid=testClusterID;nsid=469072444;c=1693717356264), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:02:36,721 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x9fcaa4ede383d3e0: Processing first storage report for DS-c93a74e1-5eeb-4fd4-aff2-8fc40a0fd788 from datanode ce93aba6-c719-4530-a1d7-3d6aa1cc0935 2023-09-03 05:02:36,721 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x9fcaa4ede383d3e0: from storage DS-c93a74e1-5eeb-4fd4-aff2-8fc40a0fd788 node DatanodeRegistration(127.0.0.1:38701, datanodeUuid=ce93aba6-c719-4530-a1d7-3d6aa1cc0935, infoPort=37119, infoSecurePort=0, ipcPort=41705, storageInfo=lv=-57;cid=testClusterID;nsid=469072444;c=1693717356264), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:02:36,752 INFO [Listener at localhost/41705] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33589 2023-09-03 05:02:36,758 WARN [Listener at localhost/36897] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-03 05:02:36,846 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x71a290df27e472c3: Processing first storage report for DS-837dec0c-fccd-4374-9397-28321ca58fa7 from datanode 3e4c6490-f7e3-47d1-ad56-9d12a38a7c8d 2023-09-03 05:02:36,846 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x71a290df27e472c3: from storage DS-837dec0c-fccd-4374-9397-28321ca58fa7 node DatanodeRegistration(127.0.0.1:36173, datanodeUuid=3e4c6490-f7e3-47d1-ad56-9d12a38a7c8d, infoPort=43011, infoSecurePort=0, ipcPort=36897, storageInfo=lv=-57;cid=testClusterID;nsid=469072444;c=1693717356264), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-03 05:02:36,846 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x71a290df27e472c3: Processing first storage report for DS-967a9aa3-21bb-4670-a126-808c66478739 from datanode 3e4c6490-f7e3-47d1-ad56-9d12a38a7c8d 2023-09-03 05:02:36,846 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x71a290df27e472c3: from storage DS-967a9aa3-21bb-4670-a126-808c66478739 node DatanodeRegistration(127.0.0.1:36173, datanodeUuid=3e4c6490-f7e3-47d1-ad56-9d12a38a7c8d, infoPort=43011, infoSecurePort=0, ipcPort=36897, storageInfo=lv=-57;cid=testClusterID;nsid=469072444;c=1693717356264), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-09-03 05:02:36,864 DEBUG [Listener at localhost/36897] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d 2023-09-03 05:02:36,867 INFO [Listener at localhost/36897] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/cluster_858f4b01-d619-9969-c868-5f1e13a4095d/zookeeper_0, clientPort=55580, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/cluster_858f4b01-d619-9969-c868-5f1e13a4095d/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/cluster_858f4b01-d619-9969-c868-5f1e13a4095d/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-09-03 05:02:36,868 INFO [Listener at localhost/36897] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=55580 2023-09-03 05:02:36,868 INFO [Listener at localhost/36897] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:02:36,869 INFO [Listener at localhost/36897] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:02:36,881 INFO [Listener at localhost/36897] util.FSUtils(471): Created version file at hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5 with version=8 2023-09-03 05:02:36,881 INFO [Listener at localhost/36897] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:39237/user/jenkins/test-data/94fe7e4c-7ad3-7453-497d-83972d419610/hbase-staging 2023-09-03 05:02:36,883 INFO [Listener at localhost/36897] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-09-03 05:02:36,883 INFO [Listener at localhost/36897] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:02:36,883 INFO [Listener at localhost/36897] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-03 05:02:36,883 INFO [Listener at localhost/36897] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-03 05:02:36,883 INFO [Listener at localhost/36897] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:02:36,883 INFO [Listener at localhost/36897] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-03 05:02:36,883 INFO [Listener at localhost/36897] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-09-03 05:02:36,884 INFO [Listener at localhost/36897] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46785 2023-09-03 05:02:36,885 INFO [Listener at localhost/36897] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:02:36,886 INFO [Listener at localhost/36897] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:02:36,887 INFO [Listener at localhost/36897] zookeeper.RecoverableZooKeeper(93): Process identifier=master:46785 connecting to ZooKeeper ensemble=127.0.0.1:55580 2023-09-03 05:02:36,897 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:467850x0, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-03 05:02:36,898 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:46785-0x100702b18e90000 connected 2023-09-03 05:02:36,914 DEBUG [Listener at localhost/36897] zookeeper.ZKUtil(164): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-03 05:02:36,915 DEBUG [Listener at localhost/36897] zookeeper.ZKUtil(164): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:02:36,915 DEBUG [Listener at localhost/36897] zookeeper.ZKUtil(164): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-03 05:02:36,916 DEBUG [Listener at localhost/36897] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46785 2023-09-03 05:02:36,916 DEBUG [Listener at localhost/36897] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46785 2023-09-03 05:02:36,916 DEBUG [Listener at localhost/36897] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46785 2023-09-03 05:02:36,916 DEBUG [Listener at localhost/36897] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46785 2023-09-03 05:02:36,917 DEBUG [Listener at localhost/36897] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46785 2023-09-03 05:02:36,917 INFO [Listener at localhost/36897] master.HMaster(444): hbase.rootdir=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5, hbase.cluster.distributed=false 2023-09-03 05:02:36,931 INFO [Listener at localhost/36897] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-09-03 05:02:36,932 INFO [Listener at localhost/36897] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:02:36,932 INFO [Listener at localhost/36897] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-03 05:02:36,932 INFO [Listener at localhost/36897] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-03 05:02:36,932 INFO [Listener at localhost/36897] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-03 05:02:36,932 INFO [Listener at localhost/36897] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-03 05:02:36,932 INFO [Listener at localhost/36897] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-09-03 05:02:36,934 INFO [Listener at localhost/36897] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36589 2023-09-03 05:02:36,935 INFO [Listener at localhost/36897] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-03 05:02:36,935 DEBUG [Listener at localhost/36897] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-09-03 05:02:36,936 INFO [Listener at localhost/36897] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:02:36,937 INFO [Listener at localhost/36897] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:02:36,938 INFO [Listener at localhost/36897] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36589 connecting to ZooKeeper ensemble=127.0.0.1:55580 2023-09-03 05:02:36,940 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): regionserver:365890x0, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-03 05:02:36,942 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36589-0x100702b18e90001 connected 2023-09-03 05:02:36,942 DEBUG [Listener at localhost/36897] zookeeper.ZKUtil(164): regionserver:36589-0x100702b18e90001, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-03 05:02:36,943 DEBUG [Listener at localhost/36897] zookeeper.ZKUtil(164): regionserver:36589-0x100702b18e90001, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:02:36,943 DEBUG [Listener at localhost/36897] zookeeper.ZKUtil(164): regionserver:36589-0x100702b18e90001, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-03 05:02:36,944 DEBUG [Listener at localhost/36897] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36589 2023-09-03 05:02:36,944 DEBUG [Listener at localhost/36897] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36589 2023-09-03 05:02:36,944 DEBUG [Listener at localhost/36897] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36589 2023-09-03 05:02:36,944 DEBUG [Listener at localhost/36897] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36589 2023-09-03 05:02:36,945 DEBUG [Listener at localhost/36897] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36589 2023-09-03 05:02:36,949 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,46785,1693717356883 2023-09-03 05:02:36,950 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-03 05:02:36,950 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,46785,1693717356883 2023-09-03 05:02:36,952 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-03 05:02:36,952 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:02:36,952 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): regionserver:36589-0x100702b18e90001, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-03 05:02:36,952 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-03 05:02:36,953 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,46785,1693717356883 from backup master directory 2023-09-03 05:02:36,953 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-03 05:02:36,955 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,46785,1693717356883 2023-09-03 05:02:36,955 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-03 05:02:36,955 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-03 05:02:36,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,46785,1693717356883 2023-09-03 05:02:36,967 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/hbase.id with ID: f773aa9b-e48d-4b9e-8092-90d12fe9c22a 2023-09-03 05:02:36,979 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:02:36,981 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:02:36,988 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x1e669682 to 127.0.0.1:55580 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-03 05:02:36,991 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2378f792, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-03 05:02:36,992 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-03 05:02:36,992 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-09-03 05:02:36,992 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-03 05:02:36,993 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/MasterData/data/master/store-tmp 2023-09-03 05:02:37,000 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:02:37,000 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-03 05:02:37,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:02:37,000 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:02:37,000 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-03 05:02:37,000 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:02:37,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:02:37,000 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-03 05:02:37,001 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/MasterData/WALs/jenkins-hbase4.apache.org,46785,1693717356883 2023-09-03 05:02:37,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46785%2C1693717356883, suffix=, logDir=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/MasterData/WALs/jenkins-hbase4.apache.org,46785,1693717356883, archiveDir=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/MasterData/oldWALs, maxLogs=10 2023-09-03 05:02:37,017 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38701,DS-4e88db21-5d8e-4fc0-a21a-5e7953c5711c,DISK] 2023-09-03 05:02:37,018 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36173,DS-837dec0c-fccd-4374-9397-28321ca58fa7,DISK] 2023-09-03 05:02:37,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/MasterData/WALs/jenkins-hbase4.apache.org,46785,1693717356883/jenkins-hbase4.apache.org%2C46785%2C1693717356883.1693717357003 2023-09-03 05:02:37,020 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38701,DS-4e88db21-5d8e-4fc0-a21a-5e7953c5711c,DISK], DatanodeInfoWithStorage[127.0.0.1:36173,DS-837dec0c-fccd-4374-9397-28321ca58fa7,DISK]] 2023-09-03 05:02:37,020 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-09-03 05:02:37,021 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:02:37,021 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:02:37,021 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:02:37,022 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:02:37,023 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-09-03 05:02:37,024 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-09-03 05:02:37,025 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:02:37,025 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:02:37,026 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:02:37,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-09-03 05:02:37,032 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-03 05:02:37,032 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=792560, jitterRate=0.007792726159095764}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-03 05:02:37,032 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-03 05:02:37,032 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-09-03 05:02:37,033 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-09-03 05:02:37,033 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-09-03 05:02:37,033 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-09-03 05:02:37,033 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-09-03 05:02:37,034 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 0 msec 2023-09-03 05:02:37,034 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-09-03 05:02:37,034 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-09-03 05:02:37,035 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-09-03 05:02:37,046 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-09-03 05:02:37,046 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-09-03 05:02:37,047 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-09-03 05:02:37,047 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-09-03 05:02:37,047 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-09-03 05:02:37,049 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:02:37,049 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-09-03 05:02:37,049 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-09-03 05:02:37,050 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-09-03 05:02:37,052 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-03 05:02:37,052 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): regionserver:36589-0x100702b18e90001, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-03 05:02:37,052 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:02:37,053 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,46785,1693717356883, sessionid=0x100702b18e90000, setting cluster-up flag (Was=false) 2023-09-03 05:02:37,056 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:02:37,060 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-09-03 05:02:37,061 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46785,1693717356883 2023-09-03 05:02:37,063 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:02:37,072 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-09-03 05:02:37,072 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46785,1693717356883 2023-09-03 05:02:37,073 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/.hbase-snapshot/.tmp 2023-09-03 05:02:37,074 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-09-03 05:02:37,074 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:02:37,074 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:02:37,074 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:02:37,074 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-03 05:02:37,074 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-09-03 05:02:37,074 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:02:37,074 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-03 05:02:37,075 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:02:37,075 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693717387075 2023-09-03 05:02:37,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-09-03 05:02:37,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-09-03 05:02:37,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-09-03 05:02:37,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-09-03 05:02:37,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-09-03 05:02:37,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-09-03 05:02:37,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-03 05:02:37,076 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-09-03 05:02:37,076 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-09-03 05:02:37,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-09-03 05:02:37,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-09-03 05:02:37,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-09-03 05:02:37,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-09-03 05:02:37,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-09-03 05:02:37,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693717357077,5,FailOnTimeoutGroup] 2023-09-03 05:02:37,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693717357077,5,FailOnTimeoutGroup] 2023-09-03 05:02:37,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-03 05:02:37,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-09-03 05:02:37,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-09-03 05:02:37,078 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-09-03 05:02:37,078 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-03 05:02:37,087 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-03 05:02:37,088 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-03 05:02:37,088 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5 2023-09-03 05:02:37,094 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:02:37,095 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-03 05:02:37,097 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/info 2023-09-03 05:02:37,097 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-03 05:02:37,098 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:02:37,098 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-03 05:02:37,099 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/rep_barrier 2023-09-03 05:02:37,099 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-03 05:02:37,100 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:02:37,100 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-03 05:02:37,101 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/table 2023-09-03 05:02:37,101 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-03 05:02:37,102 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:02:37,102 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740 2023-09-03 05:02:37,103 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740 2023-09-03 05:02:37,105 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-03 05:02:37,106 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-03 05:02:37,108 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-03 05:02:37,109 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=769111, jitterRate=-0.022026047110557556}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-03 05:02:37,109 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-03 05:02:37,109 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-03 05:02:37,109 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-03 05:02:37,109 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-03 05:02:37,109 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-03 05:02:37,109 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-03 05:02:37,109 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-03 05:02:37,109 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-03 05:02:37,110 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-09-03 05:02:37,110 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-09-03 05:02:37,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-09-03 05:02:37,111 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-09-03 05:02:37,112 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-09-03 05:02:37,146 INFO [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(951): ClusterId : f773aa9b-e48d-4b9e-8092-90d12fe9c22a 2023-09-03 05:02:37,147 DEBUG [RS:0;jenkins-hbase4:36589] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-09-03 05:02:37,149 DEBUG [RS:0;jenkins-hbase4:36589] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-09-03 05:02:37,149 DEBUG [RS:0;jenkins-hbase4:36589] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-09-03 05:02:37,152 DEBUG [RS:0;jenkins-hbase4:36589] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-09-03 05:02:37,153 DEBUG [RS:0;jenkins-hbase4:36589] zookeeper.ReadOnlyZKClient(139): Connect 0x0a6887bd to 127.0.0.1:55580 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-03 05:02:37,157 DEBUG [RS:0;jenkins-hbase4:36589] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4af6880, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-03 05:02:37,157 DEBUG [RS:0;jenkins-hbase4:36589] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@90eccd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-03 05:02:37,166 DEBUG [RS:0;jenkins-hbase4:36589] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:36589 2023-09-03 05:02:37,166 INFO [RS:0;jenkins-hbase4:36589] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-09-03 05:02:37,166 INFO [RS:0;jenkins-hbase4:36589] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-09-03 05:02:37,166 DEBUG [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(1022): About to register with Master. 2023-09-03 05:02:37,167 INFO [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,46785,1693717356883 with isa=jenkins-hbase4.apache.org/172.31.14.131:36589, startcode=1693717356931 2023-09-03 05:02:37,167 DEBUG [RS:0;jenkins-hbase4:36589] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-09-03 05:02:37,169 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44649, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-09-03 05:02:37,170 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=46785] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36589,1693717356931 2023-09-03 05:02:37,170 DEBUG [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5 2023-09-03 05:02:37,170 DEBUG [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:34487 2023-09-03 05:02:37,170 DEBUG [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-09-03 05:02:37,172 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-03 05:02:37,172 DEBUG [RS:0;jenkins-hbase4:36589] zookeeper.ZKUtil(162): regionserver:36589-0x100702b18e90001, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36589,1693717356931 2023-09-03 05:02:37,172 WARN [RS:0;jenkins-hbase4:36589] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-03 05:02:37,172 INFO [RS:0;jenkins-hbase4:36589] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-03 05:02:37,172 DEBUG [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(1948): logDir=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/WALs/jenkins-hbase4.apache.org,36589,1693717356931 2023-09-03 05:02:37,173 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36589,1693717356931] 2023-09-03 05:02:37,177 DEBUG [RS:0;jenkins-hbase4:36589] zookeeper.ZKUtil(162): regionserver:36589-0x100702b18e90001, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36589,1693717356931 2023-09-03 05:02:37,178 DEBUG [RS:0;jenkins-hbase4:36589] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-09-03 05:02:37,178 INFO [RS:0;jenkins-hbase4:36589] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-09-03 05:02:37,179 INFO [RS:0;jenkins-hbase4:36589] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-09-03 05:02:37,180 INFO [RS:0;jenkins-hbase4:36589] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-09-03 05:02:37,180 INFO [RS:0;jenkins-hbase4:36589] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-09-03 05:02:37,181 INFO [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-09-03 05:02:37,182 INFO [RS:0;jenkins-hbase4:36589] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-09-03 05:02:37,182 DEBUG [RS:0;jenkins-hbase4:36589] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:02:37,182 DEBUG [RS:0;jenkins-hbase4:36589] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:02:37,182 DEBUG [RS:0;jenkins-hbase4:36589] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:02:37,182 DEBUG [RS:0;jenkins-hbase4:36589] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:02:37,182 DEBUG [RS:0;jenkins-hbase4:36589] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:02:37,182 DEBUG [RS:0;jenkins-hbase4:36589] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-03 05:02:37,182 DEBUG [RS:0;jenkins-hbase4:36589] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:02:37,182 DEBUG [RS:0;jenkins-hbase4:36589] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:02:37,182 DEBUG [RS:0;jenkins-hbase4:36589] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:02:37,182 DEBUG [RS:0;jenkins-hbase4:36589] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-03 05:02:37,183 INFO [RS:0;jenkins-hbase4:36589] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-09-03 05:02:37,183 INFO [RS:0;jenkins-hbase4:36589] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-09-03 05:02:37,183 INFO [RS:0;jenkins-hbase4:36589] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-09-03 05:02:37,196 INFO [RS:0;jenkins-hbase4:36589] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-09-03 05:02:37,196 INFO [RS:0;jenkins-hbase4:36589] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36589,1693717356931-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-03 05:02:37,207 INFO [RS:0;jenkins-hbase4:36589] regionserver.Replication(203): jenkins-hbase4.apache.org,36589,1693717356931 started 2023-09-03 05:02:37,207 INFO [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,36589,1693717356931, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36589, sessionid=0x100702b18e90001 2023-09-03 05:02:37,207 DEBUG [RS:0;jenkins-hbase4:36589] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-09-03 05:02:37,207 DEBUG [RS:0;jenkins-hbase4:36589] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36589,1693717356931 2023-09-03 05:02:37,207 DEBUG [RS:0;jenkins-hbase4:36589] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36589,1693717356931' 2023-09-03 05:02:37,207 DEBUG [RS:0;jenkins-hbase4:36589] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-03 05:02:37,208 DEBUG [RS:0;jenkins-hbase4:36589] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-03 05:02:37,208 DEBUG [RS:0;jenkins-hbase4:36589] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-09-03 05:02:37,208 DEBUG [RS:0;jenkins-hbase4:36589] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-09-03 05:02:37,208 DEBUG [RS:0;jenkins-hbase4:36589] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36589,1693717356931 2023-09-03 05:02:37,208 DEBUG [RS:0;jenkins-hbase4:36589] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36589,1693717356931' 2023-09-03 05:02:37,208 DEBUG [RS:0;jenkins-hbase4:36589] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-09-03 05:02:37,209 DEBUG [RS:0;jenkins-hbase4:36589] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-09-03 05:02:37,209 DEBUG [RS:0;jenkins-hbase4:36589] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-09-03 05:02:37,209 INFO [RS:0;jenkins-hbase4:36589] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-09-03 05:02:37,209 INFO [RS:0;jenkins-hbase4:36589] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-09-03 05:02:37,262 DEBUG [jenkins-hbase4:46785] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-09-03 05:02:37,263 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36589,1693717356931, state=OPENING 2023-09-03 05:02:37,265 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-09-03 05:02:37,267 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:02:37,267 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-03 05:02:37,267 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36589,1693717356931}] 2023-09-03 05:02:37,311 INFO [RS:0;jenkins-hbase4:36589] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36589%2C1693717356931, suffix=, logDir=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/WALs/jenkins-hbase4.apache.org,36589,1693717356931, archiveDir=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/oldWALs, maxLogs=32 2023-09-03 05:02:37,327 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38701,DS-4e88db21-5d8e-4fc0-a21a-5e7953c5711c,DISK] 2023-09-03 05:02:37,327 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36173,DS-837dec0c-fccd-4374-9397-28321ca58fa7,DISK] 2023-09-03 05:02:37,329 INFO [RS:0;jenkins-hbase4:36589] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/WALs/jenkins-hbase4.apache.org,36589,1693717356931/jenkins-hbase4.apache.org%2C36589%2C1693717356931.1693717357312 2023-09-03 05:02:37,329 DEBUG [RS:0;jenkins-hbase4:36589] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38701,DS-4e88db21-5d8e-4fc0-a21a-5e7953c5711c,DISK], DatanodeInfoWithStorage[127.0.0.1:36173,DS-837dec0c-fccd-4374-9397-28321ca58fa7,DISK]] 2023-09-03 05:02:37,421 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36589,1693717356931 2023-09-03 05:02:37,421 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-09-03 05:02:37,423 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35754, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-09-03 05:02:37,427 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-09-03 05:02:37,427 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-03 05:02:37,428 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36589%2C1693717356931.meta, suffix=.meta, logDir=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/WALs/jenkins-hbase4.apache.org,36589,1693717356931, archiveDir=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/oldWALs, maxLogs=32 2023-09-03 05:02:37,442 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36173,DS-837dec0c-fccd-4374-9397-28321ca58fa7,DISK] 2023-09-03 05:02:37,443 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38701,DS-4e88db21-5d8e-4fc0-a21a-5e7953c5711c,DISK] 2023-09-03 05:02:37,444 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/WALs/jenkins-hbase4.apache.org,36589,1693717356931/jenkins-hbase4.apache.org%2C36589%2C1693717356931.meta.1693717357429.meta 2023-09-03 05:02:37,445 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36173,DS-837dec0c-fccd-4374-9397-28321ca58fa7,DISK], DatanodeInfoWithStorage[127.0.0.1:38701,DS-4e88db21-5d8e-4fc0-a21a-5e7953c5711c,DISK]] 2023-09-03 05:02:37,445 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-09-03 05:02:37,445 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-09-03 05:02:37,445 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-09-03 05:02:37,445 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-09-03 05:02:37,445 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-09-03 05:02:37,445 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:02:37,445 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-09-03 05:02:37,445 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-09-03 05:02:37,447 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-03 05:02:37,448 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/info 2023-09-03 05:02:37,448 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/info 2023-09-03 05:02:37,448 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-03 05:02:37,448 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:02:37,449 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-03 05:02:37,449 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/rep_barrier 2023-09-03 05:02:37,449 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/rep_barrier 2023-09-03 05:02:37,450 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-03 05:02:37,450 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:02:37,450 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-03 05:02:37,451 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/table 2023-09-03 05:02:37,451 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/table 2023-09-03 05:02:37,451 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-03 05:02:37,452 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:02:37,452 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740 2023-09-03 05:02:37,454 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740 2023-09-03 05:02:37,456 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-03 05:02:37,457 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-03 05:02:37,458 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=829125, jitterRate=0.05428798496723175}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-03 05:02:37,458 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-03 05:02:37,459 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693717357421 2023-09-03 05:02:37,462 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-09-03 05:02:37,462 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-09-03 05:02:37,463 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36589,1693717356931, state=OPEN 2023-09-03 05:02:37,464 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-09-03 05:02:37,464 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-03 05:02:37,466 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-09-03 05:02:37,466 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36589,1693717356931 in 197 msec 2023-09-03 05:02:37,467 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-09-03 05:02:37,467 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 356 msec 2023-09-03 05:02:37,468 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 395 msec 2023-09-03 05:02:37,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693717357469, completionTime=-1 2023-09-03 05:02:37,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-09-03 05:02:37,469 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-09-03 05:02:37,471 DEBUG [hconnection-0x352bcb31-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-03 05:02:37,472 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35760, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-03 05:02:37,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-09-03 05:02:37,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693717417474 2023-09-03 05:02:37,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693717477474 2023-09-03 05:02:37,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-09-03 05:02:37,479 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46785,1693717356883-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-03 05:02:37,479 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46785,1693717356883-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-03 05:02:37,479 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46785,1693717356883-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-03 05:02:37,479 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:46785, period=300000, unit=MILLISECONDS is enabled. 2023-09-03 05:02:37,479 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-09-03 05:02:37,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-09-03 05:02:37,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-03 05:02:37,481 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-09-03 05:02:37,481 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-09-03 05:02:37,482 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-09-03 05:02:37,482 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-03 05:02:37,484 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/.tmp/data/hbase/namespace/f0dc0939afba800d3bcf70783229a285 2023-09-03 05:02:37,484 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/.tmp/data/hbase/namespace/f0dc0939afba800d3bcf70783229a285 empty. 2023-09-03 05:02:37,485 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/.tmp/data/hbase/namespace/f0dc0939afba800d3bcf70783229a285 2023-09-03 05:02:37,485 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-09-03 05:02:37,495 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-09-03 05:02:37,496 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => f0dc0939afba800d3bcf70783229a285, NAME => 'hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/.tmp 2023-09-03 05:02:37,503 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:02:37,503 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing f0dc0939afba800d3bcf70783229a285, disabling compactions & flushes 2023-09-03 05:02:37,503 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285. 2023-09-03 05:02:37,504 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285. 2023-09-03 05:02:37,504 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285. after waiting 0 ms 2023-09-03 05:02:37,504 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285. 2023-09-03 05:02:37,504 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285. 2023-09-03 05:02:37,504 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for f0dc0939afba800d3bcf70783229a285: 2023-09-03 05:02:37,506 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-09-03 05:02:37,508 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693717357508"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693717357508"}]},"ts":"1693717357508"} 2023-09-03 05:02:37,510 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-03 05:02:37,510 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-03 05:02:37,510 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693717357510"}]},"ts":"1693717357510"} 2023-09-03 05:02:37,511 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-09-03 05:02:37,517 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f0dc0939afba800d3bcf70783229a285, ASSIGN}] 2023-09-03 05:02:37,519 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f0dc0939afba800d3bcf70783229a285, ASSIGN 2023-09-03 05:02:37,520 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=f0dc0939afba800d3bcf70783229a285, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36589,1693717356931; forceNewPlan=false, retain=false 2023-09-03 05:02:37,671 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f0dc0939afba800d3bcf70783229a285, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36589,1693717356931 2023-09-03 05:02:37,671 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693717357671"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693717357671"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693717357671"}]},"ts":"1693717357671"} 2023-09-03 05:02:37,672 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure f0dc0939afba800d3bcf70783229a285, server=jenkins-hbase4.apache.org,36589,1693717356931}] 2023-09-03 05:02:37,828 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285. 2023-09-03 05:02:37,828 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f0dc0939afba800d3bcf70783229a285, NAME => 'hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285.', STARTKEY => '', ENDKEY => ''} 2023-09-03 05:02:37,828 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace f0dc0939afba800d3bcf70783229a285 2023-09-03 05:02:37,828 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-03 05:02:37,829 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f0dc0939afba800d3bcf70783229a285 2023-09-03 05:02:37,829 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f0dc0939afba800d3bcf70783229a285 2023-09-03 05:02:37,830 INFO [StoreOpener-f0dc0939afba800d3bcf70783229a285-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region f0dc0939afba800d3bcf70783229a285 2023-09-03 05:02:37,831 DEBUG [StoreOpener-f0dc0939afba800d3bcf70783229a285-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/namespace/f0dc0939afba800d3bcf70783229a285/info 2023-09-03 05:02:37,831 DEBUG [StoreOpener-f0dc0939afba800d3bcf70783229a285-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/namespace/f0dc0939afba800d3bcf70783229a285/info 2023-09-03 05:02:37,831 INFO [StoreOpener-f0dc0939afba800d3bcf70783229a285-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f0dc0939afba800d3bcf70783229a285 columnFamilyName info 2023-09-03 05:02:37,832 INFO [StoreOpener-f0dc0939afba800d3bcf70783229a285-1] regionserver.HStore(310): Store=f0dc0939afba800d3bcf70783229a285/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-03 05:02:37,833 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/namespace/f0dc0939afba800d3bcf70783229a285 2023-09-03 05:02:37,834 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/namespace/f0dc0939afba800d3bcf70783229a285 2023-09-03 05:02:37,837 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f0dc0939afba800d3bcf70783229a285 2023-09-03 05:02:37,839 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/namespace/f0dc0939afba800d3bcf70783229a285/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-03 05:02:37,840 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f0dc0939afba800d3bcf70783229a285; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=850961, jitterRate=0.08205364644527435}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-03 05:02:37,840 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f0dc0939afba800d3bcf70783229a285: 2023-09-03 05:02:37,841 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285., pid=6, masterSystemTime=1693717357824 2023-09-03 05:02:37,843 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285. 2023-09-03 05:02:37,843 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285. 2023-09-03 05:02:37,844 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f0dc0939afba800d3bcf70783229a285, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36589,1693717356931 2023-09-03 05:02:37,844 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693717357844"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693717357844"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693717357844"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693717357844"}]},"ts":"1693717357844"} 2023-09-03 05:02:37,847 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-09-03 05:02:37,847 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure f0dc0939afba800d3bcf70783229a285, server=jenkins-hbase4.apache.org,36589,1693717356931 in 173 msec 2023-09-03 05:02:37,849 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-09-03 05:02:37,849 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=f0dc0939afba800d3bcf70783229a285, ASSIGN in 330 msec 2023-09-03 05:02:37,850 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-03 05:02:37,850 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693717357850"}]},"ts":"1693717357850"} 2023-09-03 05:02:37,851 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-09-03 05:02:37,853 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-09-03 05:02:37,855 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 374 msec 2023-09-03 05:02:37,882 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-09-03 05:02:37,884 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-09-03 05:02:37,884 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:02:37,887 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-09-03 05:02:37,893 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-03 05:02:37,896 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 9 msec 2023-09-03 05:02:37,898 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-09-03 05:02:37,904 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-03 05:02:37,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 8 msec 2023-09-03 05:02:37,912 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-09-03 05:02:37,916 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-09-03 05:02:37,916 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 0.961sec 2023-09-03 05:02:37,916 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-09-03 05:02:37,916 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-09-03 05:02:37,916 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-09-03 05:02:37,916 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46785,1693717356883-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-09-03 05:02:37,916 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46785,1693717356883-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-09-03 05:02:37,918 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-09-03 05:02:37,947 DEBUG [Listener at localhost/36897] zookeeper.ReadOnlyZKClient(139): Connect 0x20a22eb7 to 127.0.0.1:55580 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-03 05:02:37,955 DEBUG [Listener at localhost/36897] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@ac88460, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-03 05:02:37,960 DEBUG [hconnection-0x2716895e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-03 05:02:37,962 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35770, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-03 05:02:37,964 INFO [Listener at localhost/36897] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,46785,1693717356883 2023-09-03 05:02:37,964 INFO [Listener at localhost/36897] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-03 05:02:37,967 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-09-03 05:02:37,968 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:02:37,969 INFO [Listener at localhost/36897] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-09-03 05:02:37,969 INFO [Listener at localhost/36897] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-03 05:02:37,971 INFO [Listener at localhost/36897] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=test.com%2C8080%2C1, suffix=, logDir=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/WALs/test.com,8080,1, archiveDir=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/oldWALs, maxLogs=32 2023-09-03 05:02:37,988 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38701,DS-4e88db21-5d8e-4fc0-a21a-5e7953c5711c,DISK] 2023-09-03 05:02:37,989 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36173,DS-837dec0c-fccd-4374-9397-28321ca58fa7,DISK] 2023-09-03 05:02:37,991 INFO [Listener at localhost/36897] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/WALs/test.com,8080,1/test.com%2C8080%2C1.1693717357972 2023-09-03 05:02:37,991 DEBUG [Listener at localhost/36897] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38701,DS-4e88db21-5d8e-4fc0-a21a-5e7953c5711c,DISK], DatanodeInfoWithStorage[127.0.0.1:36173,DS-837dec0c-fccd-4374-9397-28321ca58fa7,DISK]] 2023-09-03 05:02:38,006 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38701,DS-4e88db21-5d8e-4fc0-a21a-5e7953c5711c,DISK] 2023-09-03 05:02:38,007 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36173,DS-837dec0c-fccd-4374-9397-28321ca58fa7,DISK] 2023-09-03 05:02:38,008 INFO [Listener at localhost/36897] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/WALs/test.com,8080,1/test.com%2C8080%2C1.1693717357972 with entries=0, filesize=83 B; new WAL /user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/WALs/test.com,8080,1/test.com%2C8080%2C1.1693717357992 2023-09-03 05:02:38,008 DEBUG [Listener at localhost/36897] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38701,DS-4e88db21-5d8e-4fc0-a21a-5e7953c5711c,DISK], DatanodeInfoWithStorage[127.0.0.1:36173,DS-837dec0c-fccd-4374-9397-28321ca58fa7,DISK]] 2023-09-03 05:02:38,009 DEBUG [Listener at localhost/36897] wal.AbstractFSWAL(716): hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/WALs/test.com,8080,1/test.com%2C8080%2C1.1693717357972 is not closed yet, will try archiving it next time 2023-09-03 05:02:38,015 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/WALs/test.com,8080,1/test.com%2C8080%2C1.1693717357972 to hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/oldWALs/test.com%2C8080%2C1.1693717357972 2023-09-03 05:02:38,018 DEBUG [Listener at localhost/36897] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/oldWALs 2023-09-03 05:02:38,018 INFO [Listener at localhost/36897] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL test.com%2C8080%2C1:(num 1693717357992) 2023-09-03 05:02:38,018 INFO [Listener at localhost/36897] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-09-03 05:02:38,019 DEBUG [Listener at localhost/36897] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x20a22eb7 to 127.0.0.1:55580 2023-09-03 05:02:38,019 DEBUG [Listener at localhost/36897] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:02:38,019 DEBUG [Listener at localhost/36897] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-09-03 05:02:38,019 DEBUG [Listener at localhost/36897] util.JVMClusterUtil(257): Found active master hash=1445564369, stopped=false 2023-09-03 05:02:38,019 INFO [Listener at localhost/36897] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,46785,1693717356883 2023-09-03 05:02:38,022 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-03 05:02:38,022 INFO [Listener at localhost/36897] procedure2.ProcedureExecutor(628): Stopping 2023-09-03 05:02:38,022 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:02:38,022 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): regionserver:36589-0x100702b18e90001, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-03 05:02:38,023 DEBUG [Listener at localhost/36897] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1e669682 to 127.0.0.1:55580 2023-09-03 05:02:38,023 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:02:38,023 DEBUG [Listener at localhost/36897] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:02:38,023 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36589-0x100702b18e90001, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-03 05:02:38,023 INFO [Listener at localhost/36897] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36589,1693717356931' ***** 2023-09-03 05:02:38,023 INFO [Listener at localhost/36897] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-09-03 05:02:38,024 INFO [RS:0;jenkins-hbase4:36589] regionserver.HeapMemoryManager(220): Stopping 2023-09-03 05:02:38,024 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-09-03 05:02:38,024 INFO [RS:0;jenkins-hbase4:36589] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-09-03 05:02:38,024 INFO [RS:0;jenkins-hbase4:36589] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-09-03 05:02:38,024 INFO [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(3305): Received CLOSE for f0dc0939afba800d3bcf70783229a285 2023-09-03 05:02:38,025 INFO [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36589,1693717356931 2023-09-03 05:02:38,025 DEBUG [RS:0;jenkins-hbase4:36589] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0a6887bd to 127.0.0.1:55580 2023-09-03 05:02:38,025 DEBUG [RS:0;jenkins-hbase4:36589] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:02:38,025 INFO [RS:0;jenkins-hbase4:36589] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-09-03 05:02:38,025 INFO [RS:0;jenkins-hbase4:36589] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-09-03 05:02:38,025 INFO [RS:0;jenkins-hbase4:36589] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-09-03 05:02:38,025 INFO [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-09-03 05:02:38,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f0dc0939afba800d3bcf70783229a285, disabling compactions & flushes 2023-09-03 05:02:38,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285. 2023-09-03 05:02:38,025 INFO [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-09-03 05:02:38,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285. 2023-09-03 05:02:38,026 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-03 05:02:38,025 DEBUG [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(1478): Online Regions={f0dc0939afba800d3bcf70783229a285=hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285., 1588230740=hbase:meta,,1.1588230740} 2023-09-03 05:02:38,026 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-03 05:02:38,026 DEBUG [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(1504): Waiting on 1588230740, f0dc0939afba800d3bcf70783229a285 2023-09-03 05:02:38,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285. after waiting 0 ms 2023-09-03 05:02:38,026 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-03 05:02:38,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285. 2023-09-03 05:02:38,026 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-03 05:02:38,026 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing f0dc0939afba800d3bcf70783229a285 1/1 column families, dataSize=78 B heapSize=488 B 2023-09-03 05:02:38,026 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-03 05:02:38,027 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-09-03 05:02:38,047 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/namespace/f0dc0939afba800d3bcf70783229a285/.tmp/info/155efc71d4b140e79372e4df62b4aeaf 2023-09-03 05:02:38,047 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/.tmp/info/3d839e67cf0f45c8b768e866bf415d38 2023-09-03 05:02:38,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/namespace/f0dc0939afba800d3bcf70783229a285/.tmp/info/155efc71d4b140e79372e4df62b4aeaf as hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/namespace/f0dc0939afba800d3bcf70783229a285/info/155efc71d4b140e79372e4df62b4aeaf 2023-09-03 05:02:38,060 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/namespace/f0dc0939afba800d3bcf70783229a285/info/155efc71d4b140e79372e4df62b4aeaf, entries=2, sequenceid=6, filesize=4.8 K 2023-09-03 05:02:38,062 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for f0dc0939afba800d3bcf70783229a285 in 36ms, sequenceid=6, compaction requested=false 2023-09-03 05:02:38,064 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/.tmp/table/dfc64b4765554d6eabdfbcd5116596e7 2023-09-03 05:02:38,067 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/namespace/f0dc0939afba800d3bcf70783229a285/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-09-03 05:02:38,067 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285. 2023-09-03 05:02:38,067 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f0dc0939afba800d3bcf70783229a285: 2023-09-03 05:02:38,067 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693717357480.f0dc0939afba800d3bcf70783229a285. 2023-09-03 05:02:38,069 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/.tmp/info/3d839e67cf0f45c8b768e866bf415d38 as hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/info/3d839e67cf0f45c8b768e866bf415d38 2023-09-03 05:02:38,074 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/info/3d839e67cf0f45c8b768e866bf415d38, entries=10, sequenceid=9, filesize=5.9 K 2023-09-03 05:02:38,076 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/.tmp/table/dfc64b4765554d6eabdfbcd5116596e7 as hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/table/dfc64b4765554d6eabdfbcd5116596e7 2023-09-03 05:02:38,081 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/table/dfc64b4765554d6eabdfbcd5116596e7, entries=2, sequenceid=9, filesize=4.7 K 2023-09-03 05:02:38,083 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 57ms, sequenceid=9, compaction requested=false 2023-09-03 05:02:38,089 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-09-03 05:02:38,090 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-03 05:02:38,090 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-03 05:02:38,090 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-03 05:02:38,090 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-09-03 05:02:38,226 INFO [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36589,1693717356931; all regions closed. 2023-09-03 05:02:38,232 DEBUG [RS:0;jenkins-hbase4:36589] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/oldWALs 2023-09-03 05:02:38,232 INFO [RS:0;jenkins-hbase4:36589] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36589%2C1693717356931.meta:.meta(num 1693717357429) 2023-09-03 05:02:38,242 DEBUG [RS:0;jenkins-hbase4:36589] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/oldWALs 2023-09-03 05:02:38,242 INFO [RS:0;jenkins-hbase4:36589] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36589%2C1693717356931:(num 1693717357312) 2023-09-03 05:02:38,242 DEBUG [RS:0;jenkins-hbase4:36589] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:02:38,242 INFO [RS:0;jenkins-hbase4:36589] regionserver.LeaseManager(133): Closed leases 2023-09-03 05:02:38,243 INFO [RS:0;jenkins-hbase4:36589] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-09-03 05:02:38,243 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-03 05:02:38,244 INFO [RS:0;jenkins-hbase4:36589] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36589 2023-09-03 05:02:38,247 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): regionserver:36589-0x100702b18e90001, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36589,1693717356931 2023-09-03 05:02:38,247 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-03 05:02:38,247 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): regionserver:36589-0x100702b18e90001, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-03 05:02:38,249 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36589,1693717356931] 2023-09-03 05:02:38,249 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36589,1693717356931; numProcessing=1 2023-09-03 05:02:38,250 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36589,1693717356931 already deleted, retry=false 2023-09-03 05:02:38,250 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36589,1693717356931 expired; onlineServers=0 2023-09-03 05:02:38,250 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46785,1693717356883' ***** 2023-09-03 05:02:38,250 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-09-03 05:02:38,250 DEBUG [M:0;jenkins-hbase4:46785] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@521611cc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-03 05:02:38,250 INFO [M:0;jenkins-hbase4:46785] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46785,1693717356883 2023-09-03 05:02:38,250 INFO [M:0;jenkins-hbase4:46785] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46785,1693717356883; all regions closed. 2023-09-03 05:02:38,250 DEBUG [M:0;jenkins-hbase4:46785] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-03 05:02:38,250 DEBUG [M:0;jenkins-hbase4:46785] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-09-03 05:02:38,251 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-09-03 05:02:38,251 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693717357077] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693717357077,5,FailOnTimeoutGroup] 2023-09-03 05:02:38,251 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693717357077] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693717357077,5,FailOnTimeoutGroup] 2023-09-03 05:02:38,251 DEBUG [M:0;jenkins-hbase4:46785] cleaner.HFileCleaner(317): Stopping file delete threads 2023-09-03 05:02:38,252 INFO [M:0;jenkins-hbase4:46785] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-09-03 05:02:38,252 INFO [M:0;jenkins-hbase4:46785] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-09-03 05:02:38,252 INFO [M:0;jenkins-hbase4:46785] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-09-03 05:02:38,252 DEBUG [M:0;jenkins-hbase4:46785] master.HMaster(1512): Stopping service threads 2023-09-03 05:02:38,252 INFO [M:0;jenkins-hbase4:46785] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-09-03 05:02:38,252 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-09-03 05:02:38,252 ERROR [M:0;jenkins-hbase4:46785] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] 2023-09-03 05:02:38,252 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-03 05:02:38,253 INFO [M:0;jenkins-hbase4:46785] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-09-03 05:02:38,253 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-09-03 05:02:38,253 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-03 05:02:38,253 DEBUG [M:0;jenkins-hbase4:46785] zookeeper.ZKUtil(398): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-09-03 05:02:38,253 WARN [M:0;jenkins-hbase4:46785] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-09-03 05:02:38,253 INFO [M:0;jenkins-hbase4:46785] assignment.AssignmentManager(315): Stopping assignment manager 2023-09-03 05:02:38,253 INFO [M:0;jenkins-hbase4:46785] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-09-03 05:02:38,254 DEBUG [M:0;jenkins-hbase4:46785] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-03 05:02:38,254 INFO [M:0;jenkins-hbase4:46785] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:02:38,254 DEBUG [M:0;jenkins-hbase4:46785] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:02:38,254 DEBUG [M:0;jenkins-hbase4:46785] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-03 05:02:38,254 DEBUG [M:0;jenkins-hbase4:46785] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:02:38,254 INFO [M:0;jenkins-hbase4:46785] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.07 KB heapSize=29.55 KB 2023-09-03 05:02:38,265 INFO [M:0;jenkins-hbase4:46785] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.07 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b3403dd2357546b5917bcd27409dfddb 2023-09-03 05:02:38,271 DEBUG [M:0;jenkins-hbase4:46785] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b3403dd2357546b5917bcd27409dfddb as hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b3403dd2357546b5917bcd27409dfddb 2023-09-03 05:02:38,277 INFO [M:0;jenkins-hbase4:46785] regionserver.HStore(1080): Added hdfs://localhost:34487/user/jenkins/test-data/40632d6e-59ca-652f-594f-54b87ca401e5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b3403dd2357546b5917bcd27409dfddb, entries=8, sequenceid=66, filesize=6.3 K 2023-09-03 05:02:38,277 INFO [M:0;jenkins-hbase4:46785] regionserver.HRegion(2948): Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 23ms, sequenceid=66, compaction requested=false 2023-09-03 05:02:38,279 INFO [M:0;jenkins-hbase4:46785] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-03 05:02:38,279 DEBUG [M:0;jenkins-hbase4:46785] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-03 05:02:38,287 INFO [M:0;jenkins-hbase4:46785] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-09-03 05:02:38,287 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-03 05:02:38,287 INFO [M:0;jenkins-hbase4:46785] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46785 2023-09-03 05:02:38,290 DEBUG [M:0;jenkins-hbase4:46785] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,46785,1693717356883 already deleted, retry=false 2023-09-03 05:02:38,422 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:02:38,422 INFO [M:0;jenkins-hbase4:46785] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46785,1693717356883; zookeeper connection closed. 2023-09-03 05:02:38,422 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): master:46785-0x100702b18e90000, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:02:38,522 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): regionserver:36589-0x100702b18e90001, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:02:38,522 INFO [RS:0;jenkins-hbase4:36589] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36589,1693717356931; zookeeper connection closed. 2023-09-03 05:02:38,522 DEBUG [Listener at localhost/36897-EventThread] zookeeper.ZKWatcher(600): regionserver:36589-0x100702b18e90001, quorum=127.0.0.1:55580, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-03 05:02:38,524 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@79087eaf] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@79087eaf 2023-09-03 05:02:38,524 INFO [Listener at localhost/36897] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-09-03 05:02:38,524 WARN [Listener at localhost/36897] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-03 05:02:38,528 INFO [Listener at localhost/36897] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:02:38,631 WARN [BP-2014689494-172.31.14.131-1693717356264 heartbeating to localhost/127.0.0.1:34487] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-03 05:02:38,631 WARN [BP-2014689494-172.31.14.131-1693717356264 heartbeating to localhost/127.0.0.1:34487] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2014689494-172.31.14.131-1693717356264 (Datanode Uuid 3e4c6490-f7e3-47d1-ad56-9d12a38a7c8d) service to localhost/127.0.0.1:34487 2023-09-03 05:02:38,632 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/cluster_858f4b01-d619-9969-c868-5f1e13a4095d/dfs/data/data3/current/BP-2014689494-172.31.14.131-1693717356264] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:02:38,632 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/cluster_858f4b01-d619-9969-c868-5f1e13a4095d/dfs/data/data4/current/BP-2014689494-172.31.14.131-1693717356264] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:02:38,633 WARN [Listener at localhost/36897] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-03 05:02:38,636 INFO [Listener at localhost/36897] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:02:38,739 WARN [BP-2014689494-172.31.14.131-1693717356264 heartbeating to localhost/127.0.0.1:34487] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-03 05:02:38,739 WARN [BP-2014689494-172.31.14.131-1693717356264 heartbeating to localhost/127.0.0.1:34487] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2014689494-172.31.14.131-1693717356264 (Datanode Uuid ce93aba6-c719-4530-a1d7-3d6aa1cc0935) service to localhost/127.0.0.1:34487 2023-09-03 05:02:38,740 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/cluster_858f4b01-d619-9969-c868-5f1e13a4095d/dfs/data/data1/current/BP-2014689494-172.31.14.131-1693717356264] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:02:38,740 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/428ec61a-a342-5973-b791-b3100057782d/cluster_858f4b01-d619-9969-c868-5f1e13a4095d/dfs/data/data2/current/BP-2014689494-172.31.14.131-1693717356264] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-03 05:02:38,751 INFO [Listener at localhost/36897] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-03 05:02:38,863 INFO [Listener at localhost/36897] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-09-03 05:02:38,875 INFO [Listener at localhost/36897] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-09-03 05:02:38,886 INFO [Listener at localhost/36897] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=100 (was 76) - Thread LEAK? -, OpenFileDescriptor=494 (was 464) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=31 (was 34), ProcessCount=167 (was 167), AvailableMemoryMB=8150 (was 8299)