2023-08-27 15:58:14,580 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f 2023-08-27 15:58:14,595 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.regionserver.wal.TestAsyncLogRolling timeout: 13 mins 2023-08-27 15:58:14,625 INFO [Time-limited test] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=80, ProcessCount=169, AvailableMemoryMB=12057 2023-08-27 15:58:14,631 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-27 15:58:14,632 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf, deleteOnExit=true 2023-08-27 15:58:14,632 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-27 15:58:14,632 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/test.cache.data in system properties and HBase conf 2023-08-27 15:58:14,633 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/hadoop.tmp.dir in system properties and HBase conf 2023-08-27 15:58:14,633 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/hadoop.log.dir in system properties and HBase conf 2023-08-27 15:58:14,634 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-27 15:58:14,634 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-27 15:58:14,634 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-27 15:58:14,743 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-08-27 15:58:15,145 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-27 15:58:15,149 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-27 15:58:15,150 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-27 15:58:15,150 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-27 15:58:15,150 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-27 15:58:15,151 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-27 15:58:15,151 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-27 15:58:15,151 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-27 15:58:15,152 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-27 15:58:15,152 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-27 15:58:15,152 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/nfs.dump.dir in system properties and HBase conf 2023-08-27 15:58:15,153 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/java.io.tmpdir in system properties and HBase conf 2023-08-27 15:58:15,153 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-27 15:58:15,153 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-27 15:58:15,154 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-27 15:58:15,709 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-27 15:58:15,714 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-27 15:58:16,044 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-08-27 15:58:16,211 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-08-27 15:58:16,232 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 15:58:16,270 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 15:58:16,336 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/java.io.tmpdir/Jetty_localhost_44613_hdfs____.51s35q/webapp 2023-08-27 15:58:16,482 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44613 2023-08-27 15:58:16,494 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-27 15:58:16,494 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-27 15:58:16,967 WARN [Listener at localhost/46535] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 15:58:17,066 WARN [Listener at localhost/46535] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-27 15:58:17,087 WARN [Listener at localhost/46535] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 15:58:17,093 INFO [Listener at localhost/46535] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 15:58:17,099 INFO [Listener at localhost/46535] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/java.io.tmpdir/Jetty_localhost_38831_datanode____lpavzs/webapp 2023-08-27 15:58:17,212 INFO [Listener at localhost/46535] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38831 2023-08-27 15:58:17,562 WARN [Listener at localhost/33925] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 15:58:17,588 WARN [Listener at localhost/33925] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-27 15:58:17,593 WARN [Listener at localhost/33925] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 15:58:17,595 INFO [Listener at localhost/33925] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 15:58:17,602 INFO [Listener at localhost/33925] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/java.io.tmpdir/Jetty_localhost_41329_datanode____j3l30k/webapp 2023-08-27 15:58:17,729 INFO [Listener at localhost/33925] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41329 2023-08-27 15:58:17,741 WARN [Listener at localhost/32959] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 15:58:18,107 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x5c10254ba7bf3a75: Processing first storage report for DS-8711fa39-ec58-46f3-b79c-2839adccd41b from datanode e61b5eb4-3595-469c-8729-fb2ae675ea80 2023-08-27 15:58:18,109 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x5c10254ba7bf3a75: from storage DS-8711fa39-ec58-46f3-b79c-2839adccd41b node DatanodeRegistration(127.0.0.1:37615, datanodeUuid=e61b5eb4-3595-469c-8729-fb2ae675ea80, infoPort=33439, infoSecurePort=0, ipcPort=33925, storageInfo=lv=-57;cid=testClusterID;nsid=534216922;c=1693151895798), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-08-27 15:58:18,110 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa0a6901f16f6f23b: Processing first storage report for DS-ae42adf9-a1ef-4f5d-9020-15034b1d2095 from datanode f55ee972-cd84-4088-a879-edb39a58bea0 2023-08-27 15:58:18,110 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa0a6901f16f6f23b: from storage DS-ae42adf9-a1ef-4f5d-9020-15034b1d2095 node DatanodeRegistration(127.0.0.1:36491, datanodeUuid=f55ee972-cd84-4088-a879-edb39a58bea0, infoPort=39933, infoSecurePort=0, ipcPort=32959, storageInfo=lv=-57;cid=testClusterID;nsid=534216922;c=1693151895798), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 15:58:18,110 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x5c10254ba7bf3a75: Processing first storage report for DS-6ee198f6-0e81-4aac-9225-3d549542fa1f from datanode e61b5eb4-3595-469c-8729-fb2ae675ea80 2023-08-27 15:58:18,110 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x5c10254ba7bf3a75: from storage DS-6ee198f6-0e81-4aac-9225-3d549542fa1f node DatanodeRegistration(127.0.0.1:37615, datanodeUuid=e61b5eb4-3595-469c-8729-fb2ae675ea80, infoPort=33439, infoSecurePort=0, ipcPort=33925, storageInfo=lv=-57;cid=testClusterID;nsid=534216922;c=1693151895798), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-27 15:58:18,110 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa0a6901f16f6f23b: Processing first storage report for DS-5aa8a408-eb15-4353-825d-d11471690330 from datanode f55ee972-cd84-4088-a879-edb39a58bea0 2023-08-27 15:58:18,110 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa0a6901f16f6f23b: from storage DS-5aa8a408-eb15-4353-825d-d11471690330 node DatanodeRegistration(127.0.0.1:36491, datanodeUuid=f55ee972-cd84-4088-a879-edb39a58bea0, infoPort=39933, infoSecurePort=0, ipcPort=32959, storageInfo=lv=-57;cid=testClusterID;nsid=534216922;c=1693151895798), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 15:58:18,228 DEBUG [Listener at localhost/32959] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f 2023-08-27 15:58:18,296 INFO [Listener at localhost/32959] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf/zookeeper_0, clientPort=61288, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-27 15:58:18,310 INFO [Listener at localhost/32959] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=61288 2023-08-27 15:58:18,322 INFO [Listener at localhost/32959] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:18,324 INFO [Listener at localhost/32959] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:19,015 INFO [Listener at localhost/32959] util.FSUtils(471): Created version file at hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2 with version=8 2023-08-27 15:58:19,015 INFO [Listener at localhost/32959] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/hbase-staging 2023-08-27 15:58:19,336 INFO [Listener at localhost/32959] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-08-27 15:58:19,800 INFO [Listener at localhost/32959] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-27 15:58:19,832 INFO [Listener at localhost/32959] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 15:58:19,833 INFO [Listener at localhost/32959] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-27 15:58:19,833 INFO [Listener at localhost/32959] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-27 15:58:19,833 INFO [Listener at localhost/32959] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 15:58:19,833 INFO [Listener at localhost/32959] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-27 15:58:19,983 INFO [Listener at localhost/32959] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-27 15:58:20,061 DEBUG [Listener at localhost/32959] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-08-27 15:58:20,155 INFO [Listener at localhost/32959] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38581 2023-08-27 15:58:20,165 INFO [Listener at localhost/32959] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:20,168 INFO [Listener at localhost/32959] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:20,188 INFO [Listener at localhost/32959] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38581 connecting to ZooKeeper ensemble=127.0.0.1:61288 2023-08-27 15:58:20,230 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:385810x0, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-27 15:58:20,232 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:38581-0x1004e76e24e0000 connected 2023-08-27 15:58:20,258 DEBUG [Listener at localhost/32959] zookeeper.ZKUtil(164): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-27 15:58:20,259 DEBUG [Listener at localhost/32959] zookeeper.ZKUtil(164): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 15:58:20,262 DEBUG [Listener at localhost/32959] zookeeper.ZKUtil(164): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-27 15:58:20,270 DEBUG [Listener at localhost/32959] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38581 2023-08-27 15:58:20,270 DEBUG [Listener at localhost/32959] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38581 2023-08-27 15:58:20,270 DEBUG [Listener at localhost/32959] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38581 2023-08-27 15:58:20,271 DEBUG [Listener at localhost/32959] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38581 2023-08-27 15:58:20,271 DEBUG [Listener at localhost/32959] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38581 2023-08-27 15:58:20,277 INFO [Listener at localhost/32959] master.HMaster(444): hbase.rootdir=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2, hbase.cluster.distributed=false 2023-08-27 15:58:20,357 INFO [Listener at localhost/32959] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-27 15:58:20,357 INFO [Listener at localhost/32959] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 15:58:20,357 INFO [Listener at localhost/32959] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-27 15:58:20,357 INFO [Listener at localhost/32959] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-27 15:58:20,358 INFO [Listener at localhost/32959] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 15:58:20,358 INFO [Listener at localhost/32959] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-27 15:58:20,363 INFO [Listener at localhost/32959] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-27 15:58:20,366 INFO [Listener at localhost/32959] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42391 2023-08-27 15:58:20,369 INFO [Listener at localhost/32959] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-27 15:58:20,375 DEBUG [Listener at localhost/32959] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-27 15:58:20,376 INFO [Listener at localhost/32959] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:20,379 INFO [Listener at localhost/32959] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:20,382 INFO [Listener at localhost/32959] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42391 connecting to ZooKeeper ensemble=127.0.0.1:61288 2023-08-27 15:58:20,385 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): regionserver:423910x0, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-27 15:58:20,386 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42391-0x1004e76e24e0001 connected 2023-08-27 15:58:20,386 DEBUG [Listener at localhost/32959] zookeeper.ZKUtil(164): regionserver:42391-0x1004e76e24e0001, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-27 15:58:20,388 DEBUG [Listener at localhost/32959] zookeeper.ZKUtil(164): regionserver:42391-0x1004e76e24e0001, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 15:58:20,388 DEBUG [Listener at localhost/32959] zookeeper.ZKUtil(164): regionserver:42391-0x1004e76e24e0001, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-27 15:58:20,391 DEBUG [Listener at localhost/32959] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42391 2023-08-27 15:58:20,393 DEBUG [Listener at localhost/32959] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42391 2023-08-27 15:58:20,397 DEBUG [Listener at localhost/32959] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42391 2023-08-27 15:58:20,398 DEBUG [Listener at localhost/32959] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42391 2023-08-27 15:58:20,399 DEBUG [Listener at localhost/32959] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42391 2023-08-27 15:58:20,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,38581,1693151899151 2023-08-27 15:58:20,411 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-27 15:58:20,413 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,38581,1693151899151 2023-08-27 15:58:20,436 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-27 15:58:20,436 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): regionserver:42391-0x1004e76e24e0001, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-27 15:58:20,436 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:20,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-27 15:58:20,439 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,38581,1693151899151 from backup master directory 2023-08-27 15:58:20,439 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-27 15:58:20,442 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,38581,1693151899151 2023-08-27 15:58:20,442 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-27 15:58:20,443 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-27 15:58:20,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,38581,1693151899151 2023-08-27 15:58:20,445 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-08-27 15:58:20,447 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-08-27 15:58:20,530 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/hbase.id with ID: 0ca1d4fa-4843-46c0-b79e-784800f1fce6 2023-08-27 15:58:20,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:20,590 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:20,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x11ddd9c7 to 127.0.0.1:61288 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-27 15:58:20,677 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@71cfabea, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-27 15:58:20,705 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-27 15:58:20,707 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-27 15:58:20,718 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-27 15:58:20,764 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/data/master/store-tmp 2023-08-27 15:58:20,797 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:20,798 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-27 15:58:20,798 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:58:20,798 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:58:20,798 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-27 15:58:20,798 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:58:20,798 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:58:20,798 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-27 15:58:20,800 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/WALs/jenkins-hbase4.apache.org,38581,1693151899151 2023-08-27 15:58:20,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38581%2C1693151899151, suffix=, logDir=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/WALs/jenkins-hbase4.apache.org,38581,1693151899151, archiveDir=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/oldWALs, maxLogs=10 2023-08-27 15:58:20,853 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-08-27 15:58:20,853 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-08-27 15:58:20,855 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:190) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:719) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:20,889 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:417) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:20,891 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37615,DS-8711fa39-ec58-46f3-b79c-2839adccd41b,DISK] 2023-08-27 15:58:20,891 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36491,DS-ae42adf9-a1ef-4f5d-9020-15034b1d2095,DISK] 2023-08-27 15:58:20,899 DEBUG [RS-EventLoopGroup-3-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:20,971 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/WALs/jenkins-hbase4.apache.org,38581,1693151899151/jenkins-hbase4.apache.org%2C38581%2C1693151899151.1693151900831 2023-08-27 15:58:20,972 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36491,DS-ae42adf9-a1ef-4f5d-9020-15034b1d2095,DISK], DatanodeInfoWithStorage[127.0.0.1:37615,DS-8711fa39-ec58-46f3-b79c-2839adccd41b,DISK]] 2023-08-27 15:58:20,972 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-27 15:58:20,973 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:20,976 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:58:20,977 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:58:21,037 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:58:21,046 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-27 15:58:21,073 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-27 15:58:21,085 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:21,091 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:58:21,093 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:58:21,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:58:21,114 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-27 15:58:21,115 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=873093, jitterRate=0.11019600927829742}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-27 15:58:21,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-27 15:58:21,117 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-27 15:58:21,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-27 15:58:21,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-27 15:58:21,137 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-27 15:58:21,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-08-27 15:58:21,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 31 msec 2023-08-27 15:58:21,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-27 15:58:21,194 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-27 15:58:21,199 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-27 15:58:21,223 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-27 15:58:21,227 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-27 15:58:21,229 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-27 15:58:21,234 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-27 15:58:21,239 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-27 15:58:21,242 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:21,243 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-27 15:58:21,244 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-27 15:58:21,259 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-27 15:58:21,263 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-27 15:58:21,263 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): regionserver:42391-0x1004e76e24e0001, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-27 15:58:21,263 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:21,264 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,38581,1693151899151, sessionid=0x1004e76e24e0000, setting cluster-up flag (Was=false) 2023-08-27 15:58:21,279 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:21,285 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-27 15:58:21,287 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38581,1693151899151 2023-08-27 15:58:21,293 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:21,298 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-27 15:58:21,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38581,1693151899151 2023-08-27 15:58:21,302 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/.hbase-snapshot/.tmp 2023-08-27 15:58:21,408 INFO [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(951): ClusterId : 0ca1d4fa-4843-46c0-b79e-784800f1fce6 2023-08-27 15:58:21,412 DEBUG [RS:0;jenkins-hbase4:42391] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-27 15:58:21,419 DEBUG [RS:0;jenkins-hbase4:42391] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-27 15:58:21,419 DEBUG [RS:0;jenkins-hbase4:42391] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-27 15:58:21,422 DEBUG [RS:0;jenkins-hbase4:42391] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-27 15:58:21,423 DEBUG [RS:0;jenkins-hbase4:42391] zookeeper.ReadOnlyZKClient(139): Connect 0x778ebae7 to 127.0.0.1:61288 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-27 15:58:21,425 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-27 15:58:21,428 DEBUG [RS:0;jenkins-hbase4:42391] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@79495a66, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-27 15:58:21,428 DEBUG [RS:0;jenkins-hbase4:42391] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7d76d197, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-27 15:58:21,438 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 15:58:21,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 15:58:21,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 15:58:21,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 15:58:21,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-27 15:58:21,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:21,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-27 15:58:21,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:21,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693151931444 2023-08-27 15:58:21,446 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-27 15:58:21,450 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-27 15:58:21,450 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-27 15:58:21,456 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-27 15:58:21,456 DEBUG [RS:0;jenkins-hbase4:42391] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:42391 2023-08-27 15:58:21,458 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-27 15:58:21,461 INFO [RS:0;jenkins-hbase4:42391] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-27 15:58:21,462 INFO [RS:0;jenkins-hbase4:42391] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-27 15:58:21,462 DEBUG [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(1022): About to register with Master. 2023-08-27 15:58:21,465 INFO [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38581,1693151899151 with isa=jenkins-hbase4.apache.org/172.31.14.131:42391, startcode=1693151900356 2023-08-27 15:58:21,466 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-27 15:58:21,467 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-27 15:58:21,467 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-27 15:58:21,467 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-27 15:58:21,468 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:21,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-27 15:58:21,471 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-27 15:58:21,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-27 15:58:21,475 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-27 15:58:21,475 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-27 15:58:21,478 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693151901477,5,FailOnTimeoutGroup] 2023-08-27 15:58:21,478 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693151901478,5,FailOnTimeoutGroup] 2023-08-27 15:58:21,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:21,479 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-27 15:58:21,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:21,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:21,492 DEBUG [RS:0;jenkins-hbase4:42391] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-27 15:58:21,496 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-27 15:58:21,497 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-27 15:58:21,497 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2 2023-08-27 15:58:21,514 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:21,516 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-27 15:58:21,520 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/info 2023-08-27 15:58:21,521 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-27 15:58:21,522 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:21,522 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-27 15:58:21,526 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/rep_barrier 2023-08-27 15:58:21,527 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-27 15:58:21,528 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:21,528 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-27 15:58:21,531 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/table 2023-08-27 15:58:21,531 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-27 15:58:21,532 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:21,534 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740 2023-08-27 15:58:21,535 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740 2023-08-27 15:58:21,540 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-27 15:58:21,543 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-27 15:58:21,548 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-27 15:58:21,549 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=862883, jitterRate=0.0972125381231308}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-27 15:58:21,549 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-27 15:58:21,549 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-27 15:58:21,550 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-27 15:58:21,550 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-27 15:58:21,550 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-27 15:58:21,550 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-27 15:58:21,553 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-27 15:58:21,553 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-27 15:58:21,555 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52603, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-08-27 15:58:21,559 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-27 15:58:21,559 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-27 15:58:21,569 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38581] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42391,1693151900356 2023-08-27 15:58:21,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-27 15:58:21,583 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-27 15:58:21,588 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-27 15:58:21,591 DEBUG [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2 2023-08-27 15:58:21,591 DEBUG [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:46535 2023-08-27 15:58:21,591 DEBUG [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-27 15:58:21,598 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-27 15:58:21,599 DEBUG [RS:0;jenkins-hbase4:42391] zookeeper.ZKUtil(162): regionserver:42391-0x1004e76e24e0001, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42391,1693151900356 2023-08-27 15:58:21,599 WARN [RS:0;jenkins-hbase4:42391] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-27 15:58:21,600 INFO [RS:0;jenkins-hbase4:42391] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-27 15:58:21,601 DEBUG [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(1948): logDir=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356 2023-08-27 15:58:21,602 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42391,1693151900356] 2023-08-27 15:58:21,610 DEBUG [RS:0;jenkins-hbase4:42391] zookeeper.ZKUtil(162): regionserver:42391-0x1004e76e24e0001, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42391,1693151900356 2023-08-27 15:58:21,622 DEBUG [RS:0;jenkins-hbase4:42391] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-27 15:58:21,631 INFO [RS:0;jenkins-hbase4:42391] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-27 15:58:21,652 INFO [RS:0;jenkins-hbase4:42391] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-27 15:58:21,656 INFO [RS:0;jenkins-hbase4:42391] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-27 15:58:21,657 INFO [RS:0;jenkins-hbase4:42391] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:21,658 INFO [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-27 15:58:21,666 INFO [RS:0;jenkins-hbase4:42391] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:21,667 DEBUG [RS:0;jenkins-hbase4:42391] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:21,667 DEBUG [RS:0;jenkins-hbase4:42391] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:21,667 DEBUG [RS:0;jenkins-hbase4:42391] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:21,667 DEBUG [RS:0;jenkins-hbase4:42391] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:21,668 DEBUG [RS:0;jenkins-hbase4:42391] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:21,668 DEBUG [RS:0;jenkins-hbase4:42391] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-27 15:58:21,668 DEBUG [RS:0;jenkins-hbase4:42391] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:21,668 DEBUG [RS:0;jenkins-hbase4:42391] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:21,668 DEBUG [RS:0;jenkins-hbase4:42391] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:21,668 DEBUG [RS:0;jenkins-hbase4:42391] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:21,673 INFO [RS:0;jenkins-hbase4:42391] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:21,673 INFO [RS:0;jenkins-hbase4:42391] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:21,673 INFO [RS:0;jenkins-hbase4:42391] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:21,695 INFO [RS:0;jenkins-hbase4:42391] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-27 15:58:21,697 INFO [RS:0;jenkins-hbase4:42391] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42391,1693151900356-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:21,719 INFO [RS:0;jenkins-hbase4:42391] regionserver.Replication(203): jenkins-hbase4.apache.org,42391,1693151900356 started 2023-08-27 15:58:21,719 INFO [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,42391,1693151900356, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42391, sessionid=0x1004e76e24e0001 2023-08-27 15:58:21,720 DEBUG [RS:0;jenkins-hbase4:42391] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-27 15:58:21,720 DEBUG [RS:0;jenkins-hbase4:42391] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42391,1693151900356 2023-08-27 15:58:21,720 DEBUG [RS:0;jenkins-hbase4:42391] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42391,1693151900356' 2023-08-27 15:58:21,720 DEBUG [RS:0;jenkins-hbase4:42391] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-27 15:58:21,720 DEBUG [RS:0;jenkins-hbase4:42391] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-27 15:58:21,721 DEBUG [RS:0;jenkins-hbase4:42391] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-27 15:58:21,721 DEBUG [RS:0;jenkins-hbase4:42391] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-27 15:58:21,721 DEBUG [RS:0;jenkins-hbase4:42391] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42391,1693151900356 2023-08-27 15:58:21,721 DEBUG [RS:0;jenkins-hbase4:42391] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42391,1693151900356' 2023-08-27 15:58:21,721 DEBUG [RS:0;jenkins-hbase4:42391] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-27 15:58:21,721 DEBUG [RS:0;jenkins-hbase4:42391] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-27 15:58:21,722 DEBUG [RS:0;jenkins-hbase4:42391] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-27 15:58:21,722 INFO [RS:0;jenkins-hbase4:42391] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-27 15:58:21,722 INFO [RS:0;jenkins-hbase4:42391] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-27 15:58:21,740 DEBUG [jenkins-hbase4:38581] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-27 15:58:21,742 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42391,1693151900356, state=OPENING 2023-08-27 15:58:21,749 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-27 15:58:21,750 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:21,751 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-27 15:58:21,755 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42391,1693151900356}] 2023-08-27 15:58:21,837 INFO [RS:0;jenkins-hbase4:42391] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42391%2C1693151900356, suffix=, logDir=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356, archiveDir=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/oldWALs, maxLogs=32 2023-08-27 15:58:21,862 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36491,DS-ae42adf9-a1ef-4f5d-9020-15034b1d2095,DISK] 2023-08-27 15:58:21,862 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37615,DS-8711fa39-ec58-46f3-b79c-2839adccd41b,DISK] 2023-08-27 15:58:21,867 INFO [RS:0;jenkins-hbase4:42391] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.1693151901841 2023-08-27 15:58:21,867 DEBUG [RS:0;jenkins-hbase4:42391] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36491,DS-ae42adf9-a1ef-4f5d-9020-15034b1d2095,DISK], DatanodeInfoWithStorage[127.0.0.1:37615,DS-8711fa39-ec58-46f3-b79c-2839adccd41b,DISK]] 2023-08-27 15:58:21,937 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42391,1693151900356 2023-08-27 15:58:21,940 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-27 15:58:21,942 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58350, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-27 15:58:21,955 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-27 15:58:21,956 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-27 15:58:21,959 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42391%2C1693151900356.meta, suffix=.meta, logDir=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356, archiveDir=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/oldWALs, maxLogs=32 2023-08-27 15:58:21,979 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37615,DS-8711fa39-ec58-46f3-b79c-2839adccd41b,DISK] 2023-08-27 15:58:21,980 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36491,DS-ae42adf9-a1ef-4f5d-9020-15034b1d2095,DISK] 2023-08-27 15:58:21,986 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.meta.1693151901960.meta 2023-08-27 15:58:21,989 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36491,DS-ae42adf9-a1ef-4f5d-9020-15034b1d2095,DISK], DatanodeInfoWithStorage[127.0.0.1:37615,DS-8711fa39-ec58-46f3-b79c-2839adccd41b,DISK]] 2023-08-27 15:58:21,990 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-27 15:58:21,992 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-27 15:58:22,010 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-27 15:58:22,015 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-27 15:58:22,021 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-27 15:58:22,022 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:22,022 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-27 15:58:22,022 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-27 15:58:22,025 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-27 15:58:22,027 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/info 2023-08-27 15:58:22,027 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/info 2023-08-27 15:58:22,028 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-27 15:58:22,029 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:22,029 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-27 15:58:22,031 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/rep_barrier 2023-08-27 15:58:22,031 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/rep_barrier 2023-08-27 15:58:22,032 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-27 15:58:22,033 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:22,033 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-27 15:58:22,035 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/table 2023-08-27 15:58:22,035 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/table 2023-08-27 15:58:22,035 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-27 15:58:22,036 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:22,038 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740 2023-08-27 15:58:22,041 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740 2023-08-27 15:58:22,045 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-27 15:58:22,047 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-27 15:58:22,049 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=866309, jitterRate=0.10156992077827454}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-27 15:58:22,049 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-27 15:58:22,059 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693151901930 2023-08-27 15:58:22,076 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-27 15:58:22,077 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-27 15:58:22,077 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42391,1693151900356, state=OPEN 2023-08-27 15:58:22,080 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-27 15:58:22,080 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-27 15:58:22,084 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-27 15:58:22,084 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42391,1693151900356 in 325 msec 2023-08-27 15:58:22,089 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-27 15:58:22,089 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 515 msec 2023-08-27 15:58:22,094 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 746 msec 2023-08-27 15:58:22,095 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693151902095, completionTime=-1 2023-08-27 15:58:22,095 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-27 15:58:22,096 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-27 15:58:22,157 DEBUG [hconnection-0x62240e23-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-27 15:58:22,160 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58354, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-27 15:58:22,176 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-27 15:58:22,176 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693151962176 2023-08-27 15:58:22,176 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693152022176 2023-08-27 15:58:22,176 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 80 msec 2023-08-27 15:58:22,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38581,1693151899151-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:22,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38581,1693151899151-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:22,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38581,1693151899151-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:22,204 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38581, period=300000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:22,204 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:22,210 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-27 15:58:22,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-27 15:58:22,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-27 15:58:22,232 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-27 15:58:22,234 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-27 15:58:22,236 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-27 15:58:22,258 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/.tmp/data/hbase/namespace/71ce5e4a75dd085bcfdfdb6268e391d0 2023-08-27 15:58:22,260 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/.tmp/data/hbase/namespace/71ce5e4a75dd085bcfdfdb6268e391d0 empty. 2023-08-27 15:58:22,261 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/.tmp/data/hbase/namespace/71ce5e4a75dd085bcfdfdb6268e391d0 2023-08-27 15:58:22,261 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-27 15:58:22,285 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-27 15:58:22,287 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 71ce5e4a75dd085bcfdfdb6268e391d0, NAME => 'hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/.tmp 2023-08-27 15:58:22,304 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:22,304 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 71ce5e4a75dd085bcfdfdb6268e391d0, disabling compactions & flushes 2023-08-27 15:58:22,304 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0. 2023-08-27 15:58:22,304 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0. 2023-08-27 15:58:22,304 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0. after waiting 0 ms 2023-08-27 15:58:22,304 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0. 2023-08-27 15:58:22,304 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0. 2023-08-27 15:58:22,304 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 71ce5e4a75dd085bcfdfdb6268e391d0: 2023-08-27 15:58:22,309 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-27 15:58:22,326 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693151902312"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693151902312"}]},"ts":"1693151902312"} 2023-08-27 15:58:22,354 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-27 15:58:22,356 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-27 15:58:22,361 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693151902356"}]},"ts":"1693151902356"} 2023-08-27 15:58:22,365 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-27 15:58:22,376 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=71ce5e4a75dd085bcfdfdb6268e391d0, ASSIGN}] 2023-08-27 15:58:22,379 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=71ce5e4a75dd085bcfdfdb6268e391d0, ASSIGN 2023-08-27 15:58:22,381 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=71ce5e4a75dd085bcfdfdb6268e391d0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42391,1693151900356; forceNewPlan=false, retain=false 2023-08-27 15:58:22,533 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=71ce5e4a75dd085bcfdfdb6268e391d0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42391,1693151900356 2023-08-27 15:58:22,533 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693151902532"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693151902532"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693151902532"}]},"ts":"1693151902532"} 2023-08-27 15:58:22,539 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 71ce5e4a75dd085bcfdfdb6268e391d0, server=jenkins-hbase4.apache.org,42391,1693151900356}] 2023-08-27 15:58:22,700 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0. 2023-08-27 15:58:22,701 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 71ce5e4a75dd085bcfdfdb6268e391d0, NAME => 'hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0.', STARTKEY => '', ENDKEY => ''} 2023-08-27 15:58:22,702 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 71ce5e4a75dd085bcfdfdb6268e391d0 2023-08-27 15:58:22,702 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:22,702 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 71ce5e4a75dd085bcfdfdb6268e391d0 2023-08-27 15:58:22,702 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 71ce5e4a75dd085bcfdfdb6268e391d0 2023-08-27 15:58:22,704 INFO [StoreOpener-71ce5e4a75dd085bcfdfdb6268e391d0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 71ce5e4a75dd085bcfdfdb6268e391d0 2023-08-27 15:58:22,707 DEBUG [StoreOpener-71ce5e4a75dd085bcfdfdb6268e391d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/namespace/71ce5e4a75dd085bcfdfdb6268e391d0/info 2023-08-27 15:58:22,707 DEBUG [StoreOpener-71ce5e4a75dd085bcfdfdb6268e391d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/namespace/71ce5e4a75dd085bcfdfdb6268e391d0/info 2023-08-27 15:58:22,707 INFO [StoreOpener-71ce5e4a75dd085bcfdfdb6268e391d0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 71ce5e4a75dd085bcfdfdb6268e391d0 columnFamilyName info 2023-08-27 15:58:22,708 INFO [StoreOpener-71ce5e4a75dd085bcfdfdb6268e391d0-1] regionserver.HStore(310): Store=71ce5e4a75dd085bcfdfdb6268e391d0/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:22,710 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/namespace/71ce5e4a75dd085bcfdfdb6268e391d0 2023-08-27 15:58:22,711 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/namespace/71ce5e4a75dd085bcfdfdb6268e391d0 2023-08-27 15:58:22,717 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 71ce5e4a75dd085bcfdfdb6268e391d0 2023-08-27 15:58:22,720 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/namespace/71ce5e4a75dd085bcfdfdb6268e391d0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-27 15:58:22,721 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 71ce5e4a75dd085bcfdfdb6268e391d0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=791122, jitterRate=0.0059639811515808105}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-27 15:58:22,721 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 71ce5e4a75dd085bcfdfdb6268e391d0: 2023-08-27 15:58:22,723 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0., pid=6, masterSystemTime=1693151902692 2023-08-27 15:58:22,731 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0. 2023-08-27 15:58:22,731 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0. 2023-08-27 15:58:22,733 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=71ce5e4a75dd085bcfdfdb6268e391d0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42391,1693151900356 2023-08-27 15:58:22,734 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693151902732"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693151902732"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693151902732"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693151902732"}]},"ts":"1693151902732"} 2023-08-27 15:58:22,741 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-27 15:58:22,741 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 71ce5e4a75dd085bcfdfdb6268e391d0, server=jenkins-hbase4.apache.org,42391,1693151900356 in 199 msec 2023-08-27 15:58:22,745 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-27 15:58:22,745 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=71ce5e4a75dd085bcfdfdb6268e391d0, ASSIGN in 365 msec 2023-08-27 15:58:22,746 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-27 15:58:22,747 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693151902746"}]},"ts":"1693151902746"} 2023-08-27 15:58:22,749 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-27 15:58:22,753 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-27 15:58:22,756 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 531 msec 2023-08-27 15:58:22,835 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-27 15:58:22,837 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-27 15:58:22,837 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:22,870 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-27 15:58:22,885 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-27 15:58:22,892 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 30 msec 2023-08-27 15:58:22,903 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-27 15:58:22,915 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-27 15:58:22,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-08-27 15:58:22,930 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-27 15:58:22,933 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-27 15:58:22,933 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.490sec 2023-08-27 15:58:22,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-27 15:58:22,938 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-27 15:58:22,939 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-27 15:58:22,940 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38581,1693151899151-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-27 15:58:22,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38581,1693151899151-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-27 15:58:22,954 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-27 15:58:23,012 DEBUG [Listener at localhost/32959] zookeeper.ReadOnlyZKClient(139): Connect 0x6f86fbfe to 127.0.0.1:61288 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-27 15:58:23,016 DEBUG [Listener at localhost/32959] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3b08aba7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-27 15:58:23,064 DEBUG [hconnection-0x1255a09b-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-27 15:58:23,076 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58358, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-27 15:58:23,087 INFO [Listener at localhost/32959] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,38581,1693151899151 2023-08-27 15:58:23,088 INFO [Listener at localhost/32959] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:23,096 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-27 15:58:23,096 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:23,097 INFO [Listener at localhost/32959] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-27 15:58:23,109 WARN [Listener at localhost/32959] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-27 15:58:23,113 WARN [Listener at localhost/32959] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 15:58:23,114 INFO [Listener at localhost/32959] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 15:58:23,119 INFO [Listener at localhost/32959] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/java.io.tmpdir/Jetty_localhost_39149_datanode____4nb12v/webapp 2023-08-27 15:58:23,217 INFO [Listener at localhost/32959] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39149 2023-08-27 15:58:23,228 WARN [Listener at localhost/33477] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 15:58:23,266 WARN [Listener at localhost/33477] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-27 15:58:23,270 WARN [Listener at localhost/33477] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 15:58:23,272 INFO [Listener at localhost/33477] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 15:58:23,284 INFO [Listener at localhost/33477] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/java.io.tmpdir/Jetty_localhost_43785_datanode____5jkzg/webapp 2023-08-27 15:58:23,393 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xda31dfcc8751f69f: Processing first storage report for DS-55e3eb2a-9ce9-42b6-ae76-15e547844a38 from datanode eee628a0-15ab-44c3-ab61-693169c01ef8 2023-08-27 15:58:23,393 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xda31dfcc8751f69f: from storage DS-55e3eb2a-9ce9-42b6-ae76-15e547844a38 node DatanodeRegistration(127.0.0.1:35073, datanodeUuid=eee628a0-15ab-44c3-ab61-693169c01ef8, infoPort=41441, infoSecurePort=0, ipcPort=33477, storageInfo=lv=-57;cid=testClusterID;nsid=534216922;c=1693151895798), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-27 15:58:23,393 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xda31dfcc8751f69f: Processing first storage report for DS-8758c264-73d7-43e9-99c7-97c74a0fbbd7 from datanode eee628a0-15ab-44c3-ab61-693169c01ef8 2023-08-27 15:58:23,394 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xda31dfcc8751f69f: from storage DS-8758c264-73d7-43e9-99c7-97c74a0fbbd7 node DatanodeRegistration(127.0.0.1:35073, datanodeUuid=eee628a0-15ab-44c3-ab61-693169c01ef8, infoPort=41441, infoSecurePort=0, ipcPort=33477, storageInfo=lv=-57;cid=testClusterID;nsid=534216922;c=1693151895798), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 15:58:23,441 INFO [Listener at localhost/33477] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43785 2023-08-27 15:58:23,462 WARN [Listener at localhost/36001] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 15:58:23,503 WARN [Listener at localhost/36001] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-27 15:58:23,509 WARN [Listener at localhost/36001] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 15:58:23,511 INFO [Listener at localhost/36001] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 15:58:23,515 INFO [Listener at localhost/36001] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/java.io.tmpdir/Jetty_localhost_46589_datanode____.nf2msl/webapp 2023-08-27 15:58:23,624 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x864da345efc90938: Processing first storage report for DS-19097683-8afa-4383-bd57-7d6c615bbfba from datanode 22c728fd-e725-47f4-bd30-af3055b44eb1 2023-08-27 15:58:23,624 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x864da345efc90938: from storage DS-19097683-8afa-4383-bd57-7d6c615bbfba node DatanodeRegistration(127.0.0.1:41501, datanodeUuid=22c728fd-e725-47f4-bd30-af3055b44eb1, infoPort=33969, infoSecurePort=0, ipcPort=36001, storageInfo=lv=-57;cid=testClusterID;nsid=534216922;c=1693151895798), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-27 15:58:23,624 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x864da345efc90938: Processing first storage report for DS-41299df9-4580-4c24-82ec-ad15afc2202e from datanode 22c728fd-e725-47f4-bd30-af3055b44eb1 2023-08-27 15:58:23,625 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x864da345efc90938: from storage DS-41299df9-4580-4c24-82ec-ad15afc2202e node DatanodeRegistration(127.0.0.1:41501, datanodeUuid=22c728fd-e725-47f4-bd30-af3055b44eb1, infoPort=33969, infoSecurePort=0, ipcPort=36001, storageInfo=lv=-57;cid=testClusterID;nsid=534216922;c=1693151895798), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 15:58:23,646 INFO [Listener at localhost/36001] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46589 2023-08-27 15:58:23,655 WARN [Listener at localhost/43183] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 15:58:23,758 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x33e680f51d47bdb5: Processing first storage report for DS-5ab2f4b5-8ee1-4cf2-88dc-cd60c87d324c from datanode a43b9523-4253-4b2e-9785-3b30f54e4aa1 2023-08-27 15:58:23,759 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x33e680f51d47bdb5: from storage DS-5ab2f4b5-8ee1-4cf2-88dc-cd60c87d324c node DatanodeRegistration(127.0.0.1:40335, datanodeUuid=a43b9523-4253-4b2e-9785-3b30f54e4aa1, infoPort=42951, infoSecurePort=0, ipcPort=43183, storageInfo=lv=-57;cid=testClusterID;nsid=534216922;c=1693151895798), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 15:58:23,759 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x33e680f51d47bdb5: Processing first storage report for DS-afc60643-817f-4622-9945-1f84a786635a from datanode a43b9523-4253-4b2e-9785-3b30f54e4aa1 2023-08-27 15:58:23,759 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x33e680f51d47bdb5: from storage DS-afc60643-817f-4622-9945-1f84a786635a node DatanodeRegistration(127.0.0.1:40335, datanodeUuid=a43b9523-4253-4b2e-9785-3b30f54e4aa1, infoPort=42951, infoSecurePort=0, ipcPort=43183, storageInfo=lv=-57;cid=testClusterID;nsid=534216922;c=1693151895798), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 15:58:23,778 DEBUG [Listener at localhost/43183] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-27 15:58:23,784 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38504, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-27 15:58:23,795 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38581] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-08-27 15:58:23,795 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38581] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-08-27 15:58:23,800 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38581] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-27 15:58:23,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38581] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath 2023-08-27 15:58:23,806 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_PRE_OPERATION 2023-08-27 15:58:23,808 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-27 15:58:23,810 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/014e283dd3054af7b35d1fd9da941c2a 2023-08-27 15:58:23,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38581] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRollOnDatanodeDeath" procId is: 9 2023-08-27 15:58:23,812 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/014e283dd3054af7b35d1fd9da941c2a empty. 2023-08-27 15:58:23,814 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/014e283dd3054af7b35d1fd9da941c2a 2023-08-27 15:58:23,814 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRollOnDatanodeDeath regions 2023-08-27 15:58:23,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38581] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-27 15:58:23,835 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/.tabledesc/.tableinfo.0000000001 2023-08-27 15:58:23,836 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(7675): creating {ENCODED => 014e283dd3054af7b35d1fd9da941c2a, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/.tmp 2023-08-27 15:58:23,854 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:23,854 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1604): Closing 014e283dd3054af7b35d1fd9da941c2a, disabling compactions & flushes 2023-08-27 15:58:23,854 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a. 2023-08-27 15:58:23,854 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a. 2023-08-27 15:58:23,854 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a. after waiting 0 ms 2023-08-27 15:58:23,854 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a. 2023-08-27 15:58:23,854 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a. 2023-08-27 15:58:23,854 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1558): Region close journal for 014e283dd3054af7b35d1fd9da941c2a: 2023-08-27 15:58:23,858 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ADD_TO_META 2023-08-27 15:58:23,860 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1693151903860"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693151903860"}]},"ts":"1693151903860"} 2023-08-27 15:58:23,863 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-27 15:58:23,864 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-27 15:58:23,865 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693151903864"}]},"ts":"1693151903864"} 2023-08-27 15:58:23,867 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLING in hbase:meta 2023-08-27 15:58:23,871 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=014e283dd3054af7b35d1fd9da941c2a, ASSIGN}] 2023-08-27 15:58:23,874 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=014e283dd3054af7b35d1fd9da941c2a, ASSIGN 2023-08-27 15:58:23,875 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=014e283dd3054af7b35d1fd9da941c2a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42391,1693151900356; forceNewPlan=false, retain=false 2023-08-27 15:58:24,027 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=014e283dd3054af7b35d1fd9da941c2a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42391,1693151900356 2023-08-27 15:58:24,028 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1693151904027"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693151904027"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693151904027"}]},"ts":"1693151904027"} 2023-08-27 15:58:24,032 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 014e283dd3054af7b35d1fd9da941c2a, server=jenkins-hbase4.apache.org,42391,1693151900356}] 2023-08-27 15:58:24,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a. 2023-08-27 15:58:24,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 014e283dd3054af7b35d1fd9da941c2a, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a.', STARTKEY => '', ENDKEY => ''} 2023-08-27 15:58:24,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRollOnDatanodeDeath 014e283dd3054af7b35d1fd9da941c2a 2023-08-27 15:58:24,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:24,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 014e283dd3054af7b35d1fd9da941c2a 2023-08-27 15:58:24,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 014e283dd3054af7b35d1fd9da941c2a 2023-08-27 15:58:24,196 INFO [StoreOpener-014e283dd3054af7b35d1fd9da941c2a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 014e283dd3054af7b35d1fd9da941c2a 2023-08-27 15:58:24,198 DEBUG [StoreOpener-014e283dd3054af7b35d1fd9da941c2a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/default/TestLogRolling-testLogRollOnDatanodeDeath/014e283dd3054af7b35d1fd9da941c2a/info 2023-08-27 15:58:24,198 DEBUG [StoreOpener-014e283dd3054af7b35d1fd9da941c2a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/default/TestLogRolling-testLogRollOnDatanodeDeath/014e283dd3054af7b35d1fd9da941c2a/info 2023-08-27 15:58:24,198 INFO [StoreOpener-014e283dd3054af7b35d1fd9da941c2a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 014e283dd3054af7b35d1fd9da941c2a columnFamilyName info 2023-08-27 15:58:24,199 INFO [StoreOpener-014e283dd3054af7b35d1fd9da941c2a-1] regionserver.HStore(310): Store=014e283dd3054af7b35d1fd9da941c2a/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:24,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/default/TestLogRolling-testLogRollOnDatanodeDeath/014e283dd3054af7b35d1fd9da941c2a 2023-08-27 15:58:24,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/default/TestLogRolling-testLogRollOnDatanodeDeath/014e283dd3054af7b35d1fd9da941c2a 2023-08-27 15:58:24,206 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 014e283dd3054af7b35d1fd9da941c2a 2023-08-27 15:58:24,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/default/TestLogRolling-testLogRollOnDatanodeDeath/014e283dd3054af7b35d1fd9da941c2a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-27 15:58:24,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 014e283dd3054af7b35d1fd9da941c2a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=740264, jitterRate=-0.058706119656562805}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-27 15:58:24,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 014e283dd3054af7b35d1fd9da941c2a: 2023-08-27 15:58:24,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a., pid=11, masterSystemTime=1693151904186 2023-08-27 15:58:24,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a. 2023-08-27 15:58:24,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a. 2023-08-27 15:58:24,214 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=014e283dd3054af7b35d1fd9da941c2a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42391,1693151900356 2023-08-27 15:58:24,214 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1693151904214"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693151904214"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693151904214"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693151904214"}]},"ts":"1693151904214"} 2023-08-27 15:58:24,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-27 15:58:24,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 014e283dd3054af7b35d1fd9da941c2a, server=jenkins-hbase4.apache.org,42391,1693151900356 in 185 msec 2023-08-27 15:58:24,223 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-27 15:58:24,224 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=014e283dd3054af7b35d1fd9da941c2a, ASSIGN in 349 msec 2023-08-27 15:58:24,225 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-27 15:58:24,225 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693151904225"}]},"ts":"1693151904225"} 2023-08-27 15:58:24,227 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLED in hbase:meta 2023-08-27 15:58:24,230 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_POST_OPERATION 2023-08-27 15:58:24,232 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath in 430 msec 2023-08-27 15:58:27,543 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-27 15:58:27,629 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-27 15:58:27,631 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-27 15:58:27,632 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRollOnDatanodeDeath' 2023-08-27 15:58:29,332 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-27 15:58:29,333 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-08-27 15:58:33,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38581] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-27 15:58:33,832 INFO [Listener at localhost/43183] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRollOnDatanodeDeath, procId: 9 completed 2023-08-27 15:58:33,832 DEBUG [Listener at localhost/43183] hbase.HBaseTestingUtility(3430): Waiting until all regions of table TestLogRolling-testLogRollOnDatanodeDeath get assigned. Timeout = 60000ms 2023-08-27 15:58:33,834 INFO [Listener at localhost/43183] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-08-27 15:58:33,838 INFO [Listener at localhost/43183] hbase.HBaseTestingUtility(3484): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned to meta. Checking AM states. 2023-08-27 15:58:33,839 INFO [Listener at localhost/43183] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-08-27 15:58:33,839 INFO [Listener at localhost/43183] hbase.HBaseTestingUtility(3504): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned. 2023-08-27 15:58:33,848 DEBUG [Listener at localhost/43183] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRollOnDatanodeDeath 2023-08-27 15:58:33,849 DEBUG [Listener at localhost/43183] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a. 2023-08-27 15:58:33,850 WARN [Listener at localhost/43183] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-27 15:58:33,854 INFO [Listener at localhost/43183] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 15:58:33,859 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_1640809127_17 at /127.0.0.1:55578 [Receiving block BP-1729105487-172.31.14.131-1693151895798:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:37615:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:55578 dst: /127.0.0.1:37615 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:33,859 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1799612055_17 at /127.0.0.1:55606 [Receiving block BP-1729105487-172.31.14.131-1693151895798:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:37615:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:55606 dst: /127.0.0.1:37615 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:33,859 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1799612055_17 at /127.0.0.1:55600 [Receiving block BP-1729105487-172.31.14.131-1693151895798:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:37615:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:55600 dst: /127.0.0.1:37615 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:33,957 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_1640809127_17 at /127.0.0.1:33874 [Receiving block BP-1729105487-172.31.14.131-1693151895798:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:36491:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:33874 dst: /127.0.0.1:36491 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:33,958 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1799612055_17 at /127.0.0.1:33900 [Receiving block BP-1729105487-172.31.14.131-1693151895798:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:36491:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:33900 dst: /127.0.0.1:36491 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:33,958 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1799612055_17 at /127.0.0.1:33920 [Receiving block BP-1729105487-172.31.14.131-1693151895798:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:36491:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:33920 dst: /127.0.0.1:36491 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:33,959 WARN [BP-1729105487-172.31.14.131-1693151895798 heartbeating to localhost/127.0.0.1:46535] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-27 15:58:33,960 WARN [BP-1729105487-172.31.14.131-1693151895798 heartbeating to localhost/127.0.0.1:46535] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1729105487-172.31.14.131-1693151895798 (Datanode Uuid f55ee972-cd84-4088-a879-edb39a58bea0) service to localhost/127.0.0.1:46535 2023-08-27 15:58:33,962 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf/dfs/data/data3/current/BP-1729105487-172.31.14.131-1693151895798] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:58:33,962 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf/dfs/data/data4/current/BP-1729105487-172.31.14.131-1693151895798] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:58:33,970 WARN [Listener at localhost/43183] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-27 15:58:33,974 WARN [Listener at localhost/43183] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 15:58:33,975 INFO [Listener at localhost/43183] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 15:58:33,980 INFO [Listener at localhost/43183] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/java.io.tmpdir/Jetty_localhost_44509_datanode____.aoan4r/webapp 2023-08-27 15:58:34,074 INFO [Listener at localhost/43183] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44509 2023-08-27 15:58:34,082 WARN [Listener at localhost/38365] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 15:58:34,088 WARN [AsyncFSWAL-0-hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2-prefix:jenkins-hbase4.apache.org,42391,1693151900356] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:34,090 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C42391%2C1693151900356:(num 1693151901841) roll requested 2023-08-27 15:58:34,110 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35073,DS-55e3eb2a-9ce9-42b6-ae76-15e547844a38,DISK] 2023-08-27 15:58:34,111 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40335,DS-5ab2f4b5-8ee1-4cf2-88dc-cd60c87d324c,DISK] 2023-08-27 15:58:34,117 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.1693151901841 with entries=6, filesize=3.19 KB; new WAL /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.1693151914090 2023-08-27 15:58:34,119 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35073,DS-55e3eb2a-9ce9-42b6-ae76-15e547844a38,DISK], DatanodeInfoWithStorage[127.0.0.1:40335,DS-5ab2f4b5-8ee1-4cf2-88dc-cd60c87d324c,DISK]] 2023-08-27 15:58:34,119 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.1693151901841 is not closed yet, will try archiving it next time 2023-08-27 15:58:34,120 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-27 15:58:34,120 INFO [Listener at localhost/38365] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-27 15:58:34,125 DEBUG [Listener at localhost/38365] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6f86fbfe to 127.0.0.1:61288 2023-08-27 15:58:34,125 DEBUG [Listener at localhost/38365] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 15:58:34,126 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:34,126 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:34,126 DEBUG [Listener at localhost/38365] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-27 15:58:34,127 DEBUG [Listener at localhost/38365] util.JVMClusterUtil(257): Found active master hash=576977275, stopped=false 2023-08-27 15:58:34,127 INFO [Listener at localhost/38365] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38581,1693151899151 2023-08-27 15:58:34,129 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.1693151901841 2023-08-27 15:58:34,133 WARN [IPC Server handler 2 on default port 46535] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.1693151901841 has not been closed. Lease recovery is in progress. RecoveryId = 1015 for block blk_1073741832_1008 2023-08-27 15:58:34,134 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): regionserver:42391-0x1004e76e24e0001, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-27 15:58:34,134 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-27 15:58:34,134 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:34,134 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 15:58:34,134 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42391-0x1004e76e24e0001, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 15:58:34,135 INFO [Listener at localhost/38365] procedure2.ProcedureExecutor(629): Stopping 2023-08-27 15:58:34,136 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.1693151901841 after 7ms 2023-08-27 15:58:34,137 DEBUG [Listener at localhost/38365] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x11ddd9c7 to 127.0.0.1:61288 2023-08-27 15:58:34,137 DEBUG [Listener at localhost/38365] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 15:58:34,137 INFO [Listener at localhost/38365] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42391,1693151900356' ***** 2023-08-27 15:58:34,137 INFO [Listener at localhost/38365] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-27 15:58:34,138 INFO [RS:0;jenkins-hbase4:42391] regionserver.HeapMemoryManager(220): Stopping 2023-08-27 15:58:34,138 INFO [RS:0;jenkins-hbase4:42391] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-27 15:58:34,138 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-27 15:58:34,138 INFO [RS:0;jenkins-hbase4:42391] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-27 15:58:34,139 INFO [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(3305): Received CLOSE for 71ce5e4a75dd085bcfdfdb6268e391d0 2023-08-27 15:58:34,139 INFO [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(3305): Received CLOSE for 014e283dd3054af7b35d1fd9da941c2a 2023-08-27 15:58:34,140 INFO [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42391,1693151900356 2023-08-27 15:58:34,140 DEBUG [RS:0;jenkins-hbase4:42391] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x778ebae7 to 127.0.0.1:61288 2023-08-27 15:58:34,140 DEBUG [RS:0;jenkins-hbase4:42391] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 15:58:34,140 INFO [RS:0;jenkins-hbase4:42391] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-27 15:58:34,140 INFO [RS:0;jenkins-hbase4:42391] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-27 15:58:34,140 INFO [RS:0;jenkins-hbase4:42391] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-27 15:58:34,140 INFO [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-27 15:58:34,141 INFO [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-08-27 15:58:34,141 DEBUG [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, 71ce5e4a75dd085bcfdfdb6268e391d0=hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0., 014e283dd3054af7b35d1fd9da941c2a=TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a.} 2023-08-27 15:58:34,143 DEBUG [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(1504): Waiting on 014e283dd3054af7b35d1fd9da941c2a, 1588230740, 71ce5e4a75dd085bcfdfdb6268e391d0 2023-08-27 15:58:34,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 71ce5e4a75dd085bcfdfdb6268e391d0, disabling compactions & flushes 2023-08-27 15:58:34,145 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0. 2023-08-27 15:58:34,146 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-27 15:58:34,146 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0. 2023-08-27 15:58:34,146 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-27 15:58:34,146 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-27 15:58:34,146 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0. after waiting 0 ms 2023-08-27 15:58:34,146 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-27 15:58:34,146 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0. 2023-08-27 15:58:34,146 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-27 15:58:34,147 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 71ce5e4a75dd085bcfdfdb6268e391d0 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-27 15:58:34,147 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.92 KB heapSize=5.45 KB 2023-08-27 15:58:34,159 WARN [AsyncFSWAL-0-hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2-prefix:jenkins-hbase4.apache.org,42391,1693151900356.meta] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:34,162 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C42391%2C1693151900356.meta:.meta(num 1693151901960) roll requested 2023-08-27 15:58:34,195 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41501,DS-19097683-8afa-4383-bd57-7d6c615bbfba,DISK] 2023-08-27 15:58:34,195 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40335,DS-5ab2f4b5-8ee1-4cf2-88dc-cd60c87d324c,DISK] 2023-08-27 15:58:34,203 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.meta.1693151901960.meta with entries=12, filesize=3.87 KB; new WAL /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.meta.1693151914162.meta 2023-08-27 15:58:34,204 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40335,DS-5ab2f4b5-8ee1-4cf2-88dc-cd60c87d324c,DISK], DatanodeInfoWithStorage[127.0.0.1:41501,DS-19097683-8afa-4383-bd57-7d6c615bbfba,DISK]] 2023-08-27 15:58:34,204 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.meta.1693151901960.meta is not closed yet, will try archiving it next time 2023-08-27 15:58:34,204 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:34,210 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:34,211 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.meta.1693151901960.meta 2023-08-27 15:58:34,212 WARN [IPC Server handler 0 on default port 46535] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.meta.1693151901960.meta has not been closed. Lease recovery is in progress. RecoveryId = 1017 for block blk_1073741833_1009 2023-08-27 15:58:34,212 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.meta.1693151901960.meta after 1ms 2023-08-27 15:58:34,268 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/namespace/71ce5e4a75dd085bcfdfdb6268e391d0/.tmp/info/9125572c900c479a9c37e23248f0cda4 2023-08-27 15:58:34,270 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.69 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/.tmp/info/290b1271140a4f97b91957d92e48f196 2023-08-27 15:58:34,273 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xec7a0ce0ae55f12: Processing first storage report for DS-ae42adf9-a1ef-4f5d-9020-15034b1d2095 from datanode f55ee972-cd84-4088-a879-edb39a58bea0 2023-08-27 15:58:34,274 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xec7a0ce0ae55f12: from storage DS-ae42adf9-a1ef-4f5d-9020-15034b1d2095 node DatanodeRegistration(127.0.0.1:33167, datanodeUuid=f55ee972-cd84-4088-a879-edb39a58bea0, infoPort=38531, infoSecurePort=0, ipcPort=38365, storageInfo=lv=-57;cid=testClusterID;nsid=534216922;c=1693151895798), blocks: 6, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-27 15:58:34,274 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xec7a0ce0ae55f12: Processing first storage report for DS-5aa8a408-eb15-4353-825d-d11471690330 from datanode f55ee972-cd84-4088-a879-edb39a58bea0 2023-08-27 15:58:34,275 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xec7a0ce0ae55f12: from storage DS-5aa8a408-eb15-4353-825d-d11471690330 node DatanodeRegistration(127.0.0.1:33167, datanodeUuid=f55ee972-cd84-4088-a879-edb39a58bea0, infoPort=38531, infoSecurePort=0, ipcPort=38365, storageInfo=lv=-57;cid=testClusterID;nsid=534216922;c=1693151895798), blocks: 7, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 15:58:34,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/namespace/71ce5e4a75dd085bcfdfdb6268e391d0/.tmp/info/9125572c900c479a9c37e23248f0cda4 as hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/namespace/71ce5e4a75dd085bcfdfdb6268e391d0/info/9125572c900c479a9c37e23248f0cda4 2023-08-27 15:58:34,341 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/namespace/71ce5e4a75dd085bcfdfdb6268e391d0/info/9125572c900c479a9c37e23248f0cda4, entries=2, sequenceid=6, filesize=4.8 K 2023-08-27 15:58:34,344 DEBUG [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(1504): Waiting on 014e283dd3054af7b35d1fd9da941c2a, 1588230740, 71ce5e4a75dd085bcfdfdb6268e391d0 2023-08-27 15:58:34,346 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 71ce5e4a75dd085bcfdfdb6268e391d0 in 199ms, sequenceid=6, compaction requested=false 2023-08-27 15:58:34,352 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/.tmp/table/8927149e601244fdae6b983d3aae8993 2023-08-27 15:58:34,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/namespace/71ce5e4a75dd085bcfdfdb6268e391d0/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-27 15:58:34,364 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0. 2023-08-27 15:58:34,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 71ce5e4a75dd085bcfdfdb6268e391d0: 2023-08-27 15:58:34,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693151902220.71ce5e4a75dd085bcfdfdb6268e391d0. 2023-08-27 15:58:34,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 014e283dd3054af7b35d1fd9da941c2a, disabling compactions & flushes 2023-08-27 15:58:34,365 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a. 2023-08-27 15:58:34,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a. 2023-08-27 15:58:34,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a. after waiting 0 ms 2023-08-27 15:58:34,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a. 2023-08-27 15:58:34,366 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 014e283dd3054af7b35d1fd9da941c2a 1/1 column families, dataSize=2.05 KB heapSize=2.45 KB 2023-08-27 15:58:34,366 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/.tmp/info/290b1271140a4f97b91957d92e48f196 as hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/info/290b1271140a4f97b91957d92e48f196 2023-08-27 15:58:34,379 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/info/290b1271140a4f97b91957d92e48f196, entries=20, sequenceid=14, filesize=7.4 K 2023-08-27 15:58:34,383 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/.tmp/table/8927149e601244fdae6b983d3aae8993 as hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/table/8927149e601244fdae6b983d3aae8993 2023-08-27 15:58:34,408 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.05 KB at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/default/TestLogRolling-testLogRollOnDatanodeDeath/014e283dd3054af7b35d1fd9da941c2a/.tmp/info/789a710e0b30469f97b6e7c997f2db75 2023-08-27 15:58:34,412 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/table/8927149e601244fdae6b983d3aae8993, entries=4, sequenceid=14, filesize=4.8 K 2023-08-27 15:58:34,414 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.92 KB/2994, heapSize ~5.16 KB/5288, currentSize=0 B/0 for 1588230740 in 267ms, sequenceid=14, compaction requested=false 2023-08-27 15:58:34,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/default/TestLogRolling-testLogRollOnDatanodeDeath/014e283dd3054af7b35d1fd9da941c2a/.tmp/info/789a710e0b30469f97b6e7c997f2db75 as hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/default/TestLogRolling-testLogRollOnDatanodeDeath/014e283dd3054af7b35d1fd9da941c2a/info/789a710e0b30469f97b6e7c997f2db75 2023-08-27 15:58:34,431 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/default/TestLogRolling-testLogRollOnDatanodeDeath/014e283dd3054af7b35d1fd9da941c2a/info/789a710e0b30469f97b6e7c997f2db75, entries=2, sequenceid=6, filesize=6.8 K 2023-08-27 15:58:34,434 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-08-27 15:58:34,434 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.05 KB/2102, heapSize ~2.44 KB/2496, currentSize=0 B/0 for 014e283dd3054af7b35d1fd9da941c2a in 69ms, sequenceid=6, compaction requested=false 2023-08-27 15:58:34,436 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-27 15:58:34,441 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-27 15:58:34,441 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-27 15:58:34,441 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-27 15:58:34,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/data/default/TestLogRolling-testLogRollOnDatanodeDeath/014e283dd3054af7b35d1fd9da941c2a/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-27 15:58:34,449 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a. 2023-08-27 15:58:34,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 014e283dd3054af7b35d1fd9da941c2a: 2023-08-27 15:58:34,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1693151903795.014e283dd3054af7b35d1fd9da941c2a. 2023-08-27 15:58:34,544 INFO [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42391,1693151900356; all regions closed. 2023-08-27 15:58:34,673 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-27 15:58:34,673 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-27 15:58:35,677 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-27 15:58:38,138 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.1693151901841 after 4009ms 2023-08-27 15:58:38,140 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.1693151901841 to hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/oldWALs/jenkins-hbase4.apache.org%2C42391%2C1693151900356.1693151901841 2023-08-27 15:58:38,214 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.meta.1693151901960.meta after 4003ms 2023-08-27 15:58:38,215 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/WALs/jenkins-hbase4.apache.org,42391,1693151900356/jenkins-hbase4.apache.org%2C42391%2C1693151900356.meta.1693151901960.meta to hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/oldWALs/jenkins-hbase4.apache.org%2C42391%2C1693151900356.meta.1693151901960.meta 2023-08-27 15:58:38,221 DEBUG [RS:0;jenkins-hbase4:42391] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/oldWALs 2023-08-27 15:58:38,221 INFO [RS:0;jenkins-hbase4:42391] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42391%2C1693151900356.meta:.meta(num 1693151914162) 2023-08-27 15:58:38,243 DEBUG [RS:0;jenkins-hbase4:42391] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/oldWALs 2023-08-27 15:58:38,243 INFO [RS:0;jenkins-hbase4:42391] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42391%2C1693151900356:(num 1693151914090) 2023-08-27 15:58:38,244 DEBUG [RS:0;jenkins-hbase4:42391] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 15:58:38,244 INFO [RS:0;jenkins-hbase4:42391] regionserver.LeaseManager(133): Closed leases 2023-08-27 15:58:38,244 INFO [RS:0;jenkins-hbase4:42391] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-27 15:58:38,244 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-27 15:58:38,245 INFO [RS:0;jenkins-hbase4:42391] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42391 2023-08-27 15:58:38,254 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-27 15:58:38,254 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): regionserver:42391-0x1004e76e24e0001, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42391,1693151900356 2023-08-27 15:58:38,254 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): regionserver:42391-0x1004e76e24e0001, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-27 15:58:38,255 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42391,1693151900356] 2023-08-27 15:58:38,255 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42391,1693151900356; numProcessing=1 2023-08-27 15:58:38,258 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42391,1693151900356 already deleted, retry=false 2023-08-27 15:58:38,258 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42391,1693151900356 expired; onlineServers=0 2023-08-27 15:58:38,258 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38581,1693151899151' ***** 2023-08-27 15:58:38,258 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-27 15:58:38,259 DEBUG [M:0;jenkins-hbase4:38581] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@635ef340, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-27 15:58:38,259 INFO [M:0;jenkins-hbase4:38581] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38581,1693151899151 2023-08-27 15:58:38,259 INFO [M:0;jenkins-hbase4:38581] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38581,1693151899151; all regions closed. 2023-08-27 15:58:38,259 DEBUG [M:0;jenkins-hbase4:38581] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 15:58:38,259 DEBUG [M:0;jenkins-hbase4:38581] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-27 15:58:38,260 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-27 15:58:38,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693151901477] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693151901477,5,FailOnTimeoutGroup] 2023-08-27 15:58:38,260 DEBUG [M:0;jenkins-hbase4:38581] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-27 15:58:38,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693151901478] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693151901478,5,FailOnTimeoutGroup] 2023-08-27 15:58:38,261 INFO [M:0;jenkins-hbase4:38581] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-27 15:58:38,261 INFO [M:0;jenkins-hbase4:38581] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-27 15:58:38,262 INFO [M:0;jenkins-hbase4:38581] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-27 15:58:38,262 DEBUG [M:0;jenkins-hbase4:38581] master.HMaster(1512): Stopping service threads 2023-08-27 15:58:38,262 INFO [M:0;jenkins-hbase4:38581] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-27 15:58:38,263 ERROR [M:0;jenkins-hbase4:38581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] 2023-08-27 15:58:38,264 INFO [M:0;jenkins-hbase4:38581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-27 15:58:38,264 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-27 15:58:38,264 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-27 15:58:38,264 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:38,264 DEBUG [M:0;jenkins-hbase4:38581] zookeeper.ZKUtil(398): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-27 15:58:38,265 WARN [M:0;jenkins-hbase4:38581] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-27 15:58:38,265 INFO [M:0;jenkins-hbase4:38581] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-27 15:58:38,265 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-27 15:58:38,265 INFO [M:0;jenkins-hbase4:38581] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-27 15:58:38,266 DEBUG [M:0;jenkins-hbase4:38581] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-27 15:58:38,266 INFO [M:0;jenkins-hbase4:38581] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:58:38,266 DEBUG [M:0;jenkins-hbase4:38581] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:58:38,266 DEBUG [M:0;jenkins-hbase4:38581] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-27 15:58:38,266 DEBUG [M:0;jenkins-hbase4:38581] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:58:38,266 INFO [M:0;jenkins-hbase4:38581] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.08 KB heapSize=45.73 KB 2023-08-27 15:58:38,267 WARN [AsyncFSWAL-0-hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData-prefix:jenkins-hbase4.apache.org,38581,1693151899151] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:38,267 DEBUG [master:store-WAL-Roller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C38581%2C1693151899151:(num 1693151900831) roll requested 2023-08-27 15:58:38,291 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33167,DS-ae42adf9-a1ef-4f5d-9020-15034b1d2095,DISK] 2023-08-27 15:58:38,292 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41501,DS-19097683-8afa-4383-bd57-7d6c615bbfba,DISK] 2023-08-27 15:58:38,296 INFO [master:store-WAL-Roller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/WALs/jenkins-hbase4.apache.org,38581,1693151899151/jenkins-hbase4.apache.org%2C38581%2C1693151899151.1693151900831 with entries=89, filesize=43.93 KB; new WAL /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/WALs/jenkins-hbase4.apache.org,38581,1693151899151/jenkins-hbase4.apache.org%2C38581%2C1693151899151.1693151918267 2023-08-27 15:58:38,296 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41501,DS-19097683-8afa-4383-bd57-7d6c615bbfba,DISK], DatanodeInfoWithStorage[127.0.0.1:33167,DS-ae42adf9-a1ef-4f5d-9020-15034b1d2095,DISK]] 2023-08-27 15:58:38,297 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(716): hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/WALs/jenkins-hbase4.apache.org,38581,1693151899151/jenkins-hbase4.apache.org%2C38581%2C1693151899151.1693151900831 is not closed yet, will try archiving it next time 2023-08-27 15:58:38,297 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:38,297 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-27 15:58:38,297 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/WALs/jenkins-hbase4.apache.org,38581,1693151899151/jenkins-hbase4.apache.org%2C38581%2C1693151899151.1693151900831 2023-08-27 15:58:38,301 WARN [IPC Server handler 2 on default port 46535] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/WALs/jenkins-hbase4.apache.org,38581,1693151899151/jenkins-hbase4.apache.org%2C38581%2C1693151899151.1693151900831 has not been closed. Lease recovery is in progress. RecoveryId = 1023 for block blk_1073741829_1005 2023-08-27 15:58:38,302 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/WALs/jenkins-hbase4.apache.org,38581,1693151899151/jenkins-hbase4.apache.org%2C38581%2C1693151899151.1693151900831 after 5ms 2023-08-27 15:58:38,319 INFO [M:0;jenkins-hbase4:38581] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.08 KB at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e7ec072bf7f5472499e28c3b25c60a84 2023-08-27 15:58:38,329 DEBUG [M:0;jenkins-hbase4:38581] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e7ec072bf7f5472499e28c3b25c60a84 as hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e7ec072bf7f5472499e28c3b25c60a84 2023-08-27 15:58:38,339 INFO [M:0;jenkins-hbase4:38581] regionserver.HStore(1080): Added hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e7ec072bf7f5472499e28c3b25c60a84, entries=11, sequenceid=91, filesize=7.0 K 2023-08-27 15:58:38,341 INFO [M:0;jenkins-hbase4:38581] regionserver.HRegion(2948): Finished flush of dataSize ~38.08 KB/38997, heapSize ~45.72 KB/46816, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 74ms, sequenceid=91, compaction requested=false 2023-08-27 15:58:38,342 INFO [M:0;jenkins-hbase4:38581] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:58:38,342 DEBUG [M:0;jenkins-hbase4:38581] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-27 15:58:38,347 WARN [Close-WAL-Writer-1] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/WALs/jenkins-hbase4.apache.org,38581,1693151899151/jenkins-hbase4.apache.org%2C38581%2C1693151899151.1693151918267 not finished, retry = 0 2023-08-27 15:58:38,358 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): regionserver:42391-0x1004e76e24e0001, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 15:58:38,358 INFO [RS:0;jenkins-hbase4:42391] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42391,1693151900356; zookeeper connection closed. 2023-08-27 15:58:38,358 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): regionserver:42391-0x1004e76e24e0001, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 15:58:38,358 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4dda6b63] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4dda6b63 2023-08-27 15:58:38,359 INFO [Listener at localhost/38365] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-27 15:58:39,332 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-27 15:58:39,906 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-27 15:58:42,303 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/WALs/jenkins-hbase4.apache.org,38581,1693151899151/jenkins-hbase4.apache.org%2C38581%2C1693151899151.1693151900831 after 4006ms 2023-08-27 15:58:42,304 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/WALs/jenkins-hbase4.apache.org,38581,1693151899151/jenkins-hbase4.apache.org%2C38581%2C1693151899151.1693151900831 to hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/oldWALs/jenkins-hbase4.apache.org%2C38581%2C1693151899151.1693151900831 2023-08-27 15:58:42,311 INFO [WAL-Archive-0] region.MasterRegionUtils(50): Moved hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/MasterData/oldWALs/jenkins-hbase4.apache.org%2C38581%2C1693151899151.1693151900831 to hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/oldWALs/jenkins-hbase4.apache.org%2C38581%2C1693151899151.1693151900831$masterlocalwal$ 2023-08-27 15:58:42,311 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-27 15:58:42,312 INFO [M:0;jenkins-hbase4:38581] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-27 15:58:42,312 INFO [M:0;jenkins-hbase4:38581] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38581 2023-08-27 15:58:42,314 DEBUG [M:0;jenkins-hbase4:38581] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,38581,1693151899151 already deleted, retry=false 2023-08-27 15:58:42,417 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 15:58:42,417 INFO [M:0;jenkins-hbase4:38581] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38581,1693151899151; zookeeper connection closed. 2023-08-27 15:58:42,418 DEBUG [Listener at localhost/32959-EventThread] zookeeper.ZKWatcher(600): master:38581-0x1004e76e24e0000, quorum=127.0.0.1:61288, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 15:58:42,420 WARN [Listener at localhost/38365] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-27 15:58:42,425 INFO [Listener at localhost/38365] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 15:58:42,530 WARN [BP-1729105487-172.31.14.131-1693151895798 heartbeating to localhost/127.0.0.1:46535] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-27 15:58:42,530 WARN [BP-1729105487-172.31.14.131-1693151895798 heartbeating to localhost/127.0.0.1:46535] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1729105487-172.31.14.131-1693151895798 (Datanode Uuid f55ee972-cd84-4088-a879-edb39a58bea0) service to localhost/127.0.0.1:46535 2023-08-27 15:58:42,530 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf/dfs/data/data3/current/BP-1729105487-172.31.14.131-1693151895798] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:58:42,531 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf/dfs/data/data4/current/BP-1729105487-172.31.14.131-1693151895798] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:58:42,533 WARN [Listener at localhost/38365] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-27 15:58:42,537 INFO [Listener at localhost/38365] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 15:58:42,640 WARN [BP-1729105487-172.31.14.131-1693151895798 heartbeating to localhost/127.0.0.1:46535] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-27 15:58:42,640 WARN [BP-1729105487-172.31.14.131-1693151895798 heartbeating to localhost/127.0.0.1:46535] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1729105487-172.31.14.131-1693151895798 (Datanode Uuid a43b9523-4253-4b2e-9785-3b30f54e4aa1) service to localhost/127.0.0.1:46535 2023-08-27 15:58:42,641 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf/dfs/data/data9/current/BP-1729105487-172.31.14.131-1693151895798] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:58:42,641 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf/dfs/data/data10/current/BP-1729105487-172.31.14.131-1693151895798] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:58:42,643 WARN [Listener at localhost/38365] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-27 15:58:42,651 INFO [Listener at localhost/38365] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 15:58:42,755 WARN [BP-1729105487-172.31.14.131-1693151895798 heartbeating to localhost/127.0.0.1:46535] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-27 15:58:42,756 WARN [BP-1729105487-172.31.14.131-1693151895798 heartbeating to localhost/127.0.0.1:46535] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1729105487-172.31.14.131-1693151895798 (Datanode Uuid 22c728fd-e725-47f4-bd30-af3055b44eb1) service to localhost/127.0.0.1:46535 2023-08-27 15:58:42,757 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf/dfs/data/data7/current/BP-1729105487-172.31.14.131-1693151895798] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:58:42,757 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf/dfs/data/data8/current/BP-1729105487-172.31.14.131-1693151895798] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:58:42,758 WARN [Listener at localhost/38365] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-27 15:58:42,764 INFO [Listener at localhost/38365] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 15:58:42,867 WARN [BP-1729105487-172.31.14.131-1693151895798 heartbeating to localhost/127.0.0.1:46535] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-27 15:58:42,867 WARN [BP-1729105487-172.31.14.131-1693151895798 heartbeating to localhost/127.0.0.1:46535] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1729105487-172.31.14.131-1693151895798 (Datanode Uuid eee628a0-15ab-44c3-ab61-693169c01ef8) service to localhost/127.0.0.1:46535 2023-08-27 15:58:42,868 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf/dfs/data/data5/current/BP-1729105487-172.31.14.131-1693151895798] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:58:42,868 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf/dfs/data/data6/current/BP-1729105487-172.31.14.131-1693151895798] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:58:42,869 WARN [Listener at localhost/38365] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-27 15:58:42,871 INFO [Listener at localhost/38365] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 15:58:42,972 WARN [BP-1729105487-172.31.14.131-1693151895798 heartbeating to localhost/127.0.0.1:46535] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-27 15:58:42,972 WARN [BP-1729105487-172.31.14.131-1693151895798 heartbeating to localhost/127.0.0.1:46535] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1729105487-172.31.14.131-1693151895798 (Datanode Uuid e61b5eb4-3595-469c-8729-fb2ae675ea80) service to localhost/127.0.0.1:46535 2023-08-27 15:58:42,973 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf/dfs/data/data1/current/BP-1729105487-172.31.14.131-1693151895798] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:58:42,973 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/cluster_54ebe6e8-94f3-8b68-0006-b27dd0f93ebf/dfs/data/data2/current/BP-1729105487-172.31.14.131-1693151895798] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:58:43,006 INFO [Listener at localhost/38365] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 15:58:43,123 INFO [Listener at localhost/38365] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-27 15:58:43,173 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-27 15:58:43,191 INFO [Listener at localhost/38365] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=78 (was 10) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/38365 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:46535 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (524694476) connection to localhost/127.0.0.1:46535 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (524694476) connection to localhost/127.0.0.1:46535 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@71a8e5a6 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (524694476) connection to localhost/127.0.0.1:46535 from jenkins.hfs.0 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:46535 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) - Thread LEAK? -, OpenFileDescriptor=467 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=156 (was 80) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=12148 (was 12057) - AvailableMemoryMB LEAK? - 2023-08-27 15:58:43,207 INFO [Listener at localhost/38365] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=78, OpenFileDescriptor=467, MaxFileDescriptor=60000, SystemLoadAverage=156, ProcessCount=169, AvailableMemoryMB=12147 2023-08-27 15:58:43,208 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-27 15:58:43,208 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/hadoop.log.dir so I do NOT create it in target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8 2023-08-27 15:58:43,208 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/81d00cef-36cb-8932-22a5-63d2f263e99f/hadoop.tmp.dir so I do NOT create it in target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8 2023-08-27 15:58:43,208 INFO [Listener at localhost/38365] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/cluster_7303c16a-7635-dfea-a971-3a5fd5f1b5c3, deleteOnExit=true 2023-08-27 15:58:43,208 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-27 15:58:43,209 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/test.cache.data in system properties and HBase conf 2023-08-27 15:58:43,209 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/hadoop.tmp.dir in system properties and HBase conf 2023-08-27 15:58:43,209 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/hadoop.log.dir in system properties and HBase conf 2023-08-27 15:58:43,209 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-27 15:58:43,209 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-27 15:58:43,209 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-27 15:58:43,209 DEBUG [Listener at localhost/38365] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-27 15:58:43,210 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-27 15:58:43,210 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-27 15:58:43,210 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-27 15:58:43,210 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-27 15:58:43,211 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-27 15:58:43,211 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-27 15:58:43,211 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-27 15:58:43,211 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-27 15:58:43,211 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-27 15:58:43,211 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/nfs.dump.dir in system properties and HBase conf 2023-08-27 15:58:43,211 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/java.io.tmpdir in system properties and HBase conf 2023-08-27 15:58:43,211 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-27 15:58:43,211 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-27 15:58:43,212 INFO [Listener at localhost/38365] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-27 15:58:43,219 WARN [Listener at localhost/38365] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-27 15:58:43,219 WARN [Listener at localhost/38365] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-27 15:58:43,281 WARN [Listener at localhost/38365] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 15:58:43,284 INFO [Listener at localhost/38365] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 15:58:43,290 INFO [Listener at localhost/38365] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/java.io.tmpdir/Jetty_localhost_41201_hdfs____se3cu8/webapp 2023-08-27 15:58:43,392 INFO [Listener at localhost/38365] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41201 2023-08-27 15:58:43,397 WARN [Listener at localhost/38365] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-27 15:58:43,397 WARN [Listener at localhost/38365] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-27 15:58:43,460 WARN [Listener at localhost/36261] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 15:58:43,481 WARN [Listener at localhost/36261] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-27 15:58:43,484 WARN [Listener at localhost/36261] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 15:58:43,486 INFO [Listener at localhost/36261] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 15:58:43,495 INFO [Listener at localhost/36261] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/java.io.tmpdir/Jetty_localhost_34877_datanode____.72yema/webapp 2023-08-27 15:58:43,595 INFO [Listener at localhost/36261] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34877 2023-08-27 15:58:43,609 WARN [Listener at localhost/35785] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 15:58:43,621 WARN [Listener at localhost/35785] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-27 15:58:43,624 WARN [Listener at localhost/35785] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 15:58:43,625 INFO [Listener at localhost/35785] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 15:58:43,629 INFO [Listener at localhost/35785] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/java.io.tmpdir/Jetty_localhost_37693_datanode____.8k37sd/webapp 2023-08-27 15:58:43,704 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x7f03ffd698777a03: Processing first storage report for DS-a11783fe-967d-4589-a49d-d8f189794513 from datanode 0ca1b14c-0312-4ed1-afda-aeac18008fb4 2023-08-27 15:58:43,704 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x7f03ffd698777a03: from storage DS-a11783fe-967d-4589-a49d-d8f189794513 node DatanodeRegistration(127.0.0.1:37545, datanodeUuid=0ca1b14c-0312-4ed1-afda-aeac18008fb4, infoPort=35013, infoSecurePort=0, ipcPort=35785, storageInfo=lv=-57;cid=testClusterID;nsid=1592461411;c=1693151923225), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-27 15:58:43,705 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x7f03ffd698777a03: Processing first storage report for DS-c280adb2-5bc9-46c7-9b50-46c5954fbd8c from datanode 0ca1b14c-0312-4ed1-afda-aeac18008fb4 2023-08-27 15:58:43,705 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x7f03ffd698777a03: from storage DS-c280adb2-5bc9-46c7-9b50-46c5954fbd8c node DatanodeRegistration(127.0.0.1:37545, datanodeUuid=0ca1b14c-0312-4ed1-afda-aeac18008fb4, infoPort=35013, infoSecurePort=0, ipcPort=35785, storageInfo=lv=-57;cid=testClusterID;nsid=1592461411;c=1693151923225), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 15:58:43,733 INFO [Listener at localhost/35785] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37693 2023-08-27 15:58:43,741 WARN [Listener at localhost/40339] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 15:58:43,862 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x660f9fa3bae4f01d: Processing first storage report for DS-b140b406-079e-4601-935e-db73bafb4d8e from datanode 56abea07-b38c-4926-a0a9-d545a435eda9 2023-08-27 15:58:43,862 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x660f9fa3bae4f01d: from storage DS-b140b406-079e-4601-935e-db73bafb4d8e node DatanodeRegistration(127.0.0.1:42655, datanodeUuid=56abea07-b38c-4926-a0a9-d545a435eda9, infoPort=45679, infoSecurePort=0, ipcPort=40339, storageInfo=lv=-57;cid=testClusterID;nsid=1592461411;c=1693151923225), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 15:58:43,862 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x660f9fa3bae4f01d: Processing first storage report for DS-6848d187-35d3-4f5c-8010-4c7ea05da11d from datanode 56abea07-b38c-4926-a0a9-d545a435eda9 2023-08-27 15:58:43,862 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x660f9fa3bae4f01d: from storage DS-6848d187-35d3-4f5c-8010-4c7ea05da11d node DatanodeRegistration(127.0.0.1:42655, datanodeUuid=56abea07-b38c-4926-a0a9-d545a435eda9, infoPort=45679, infoSecurePort=0, ipcPort=40339, storageInfo=lv=-57;cid=testClusterID;nsid=1592461411;c=1693151923225), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 15:58:43,949 DEBUG [Listener at localhost/40339] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8 2023-08-27 15:58:43,952 INFO [Listener at localhost/40339] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/cluster_7303c16a-7635-dfea-a971-3a5fd5f1b5c3/zookeeper_0, clientPort=62874, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/cluster_7303c16a-7635-dfea-a971-3a5fd5f1b5c3/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/cluster_7303c16a-7635-dfea-a971-3a5fd5f1b5c3/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-27 15:58:43,954 INFO [Listener at localhost/40339] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=62874 2023-08-27 15:58:43,955 INFO [Listener at localhost/40339] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:43,956 INFO [Listener at localhost/40339] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:43,971 INFO [Listener at localhost/40339] util.FSUtils(471): Created version file at hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18 with version=8 2023-08-27 15:58:43,972 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/hbase-staging 2023-08-27 15:58:43,974 INFO [Listener at localhost/40339] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-27 15:58:43,974 INFO [Listener at localhost/40339] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 15:58:43,974 INFO [Listener at localhost/40339] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-27 15:58:43,974 INFO [Listener at localhost/40339] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-27 15:58:43,974 INFO [Listener at localhost/40339] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 15:58:43,974 INFO [Listener at localhost/40339] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-27 15:58:43,975 INFO [Listener at localhost/40339] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-27 15:58:43,976 INFO [Listener at localhost/40339] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40183 2023-08-27 15:58:43,976 INFO [Listener at localhost/40339] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:43,978 INFO [Listener at localhost/40339] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:43,979 INFO [Listener at localhost/40339] zookeeper.RecoverableZooKeeper(93): Process identifier=master:40183 connecting to ZooKeeper ensemble=127.0.0.1:62874 2023-08-27 15:58:43,986 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:401830x0, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-27 15:58:43,987 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:40183-0x1004e7746960000 connected 2023-08-27 15:58:44,001 DEBUG [Listener at localhost/40339] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-27 15:58:44,002 DEBUG [Listener at localhost/40339] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 15:58:44,003 DEBUG [Listener at localhost/40339] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-27 15:58:44,003 DEBUG [Listener at localhost/40339] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40183 2023-08-27 15:58:44,003 DEBUG [Listener at localhost/40339] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40183 2023-08-27 15:58:44,003 DEBUG [Listener at localhost/40339] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40183 2023-08-27 15:58:44,004 DEBUG [Listener at localhost/40339] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40183 2023-08-27 15:58:44,004 DEBUG [Listener at localhost/40339] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40183 2023-08-27 15:58:44,004 INFO [Listener at localhost/40339] master.HMaster(444): hbase.rootdir=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18, hbase.cluster.distributed=false 2023-08-27 15:58:44,017 INFO [Listener at localhost/40339] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-27 15:58:44,018 INFO [Listener at localhost/40339] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 15:58:44,018 INFO [Listener at localhost/40339] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-27 15:58:44,018 INFO [Listener at localhost/40339] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-27 15:58:44,018 INFO [Listener at localhost/40339] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 15:58:44,018 INFO [Listener at localhost/40339] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-27 15:58:44,018 INFO [Listener at localhost/40339] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-27 15:58:44,020 INFO [Listener at localhost/40339] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38379 2023-08-27 15:58:44,021 INFO [Listener at localhost/40339] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-27 15:58:44,022 DEBUG [Listener at localhost/40339] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-27 15:58:44,022 INFO [Listener at localhost/40339] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:44,024 INFO [Listener at localhost/40339] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:44,025 INFO [Listener at localhost/40339] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38379 connecting to ZooKeeper ensemble=127.0.0.1:62874 2023-08-27 15:58:44,029 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:383790x0, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-27 15:58:44,030 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:38379-0x1004e7746960001 connected 2023-08-27 15:58:44,030 DEBUG [Listener at localhost/40339] zookeeper.ZKUtil(164): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-27 15:58:44,030 DEBUG [Listener at localhost/40339] zookeeper.ZKUtil(164): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 15:58:44,031 DEBUG [Listener at localhost/40339] zookeeper.ZKUtil(164): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-27 15:58:44,031 DEBUG [Listener at localhost/40339] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38379 2023-08-27 15:58:44,031 DEBUG [Listener at localhost/40339] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38379 2023-08-27 15:58:44,034 DEBUG [Listener at localhost/40339] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38379 2023-08-27 15:58:44,035 DEBUG [Listener at localhost/40339] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38379 2023-08-27 15:58:44,035 DEBUG [Listener at localhost/40339] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38379 2023-08-27 15:58:44,037 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,40183,1693151923973 2023-08-27 15:58:44,039 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-27 15:58:44,039 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,40183,1693151923973 2023-08-27 15:58:44,040 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-27 15:58:44,041 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-27 15:58:44,041 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:44,041 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-27 15:58:44,042 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,40183,1693151923973 from backup master directory 2023-08-27 15:58:44,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-27 15:58:44,044 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,40183,1693151923973 2023-08-27 15:58:44,044 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-27 15:58:44,044 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-27 15:58:44,044 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,40183,1693151923973 2023-08-27 15:58:44,066 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/hbase.id with ID: f914143f-d85e-453f-a298-35d945e11255 2023-08-27 15:58:44,081 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:44,086 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:44,096 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x73588b3e to 127.0.0.1:62874 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-27 15:58:44,101 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@34f9b361, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-27 15:58:44,102 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-27 15:58:44,103 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-27 15:58:44,103 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-27 15:58:44,104 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/MasterData/data/master/store-tmp 2023-08-27 15:58:44,115 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:44,115 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-27 15:58:44,115 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:58:44,115 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:58:44,115 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-27 15:58:44,115 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:58:44,115 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:58:44,115 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-27 15:58:44,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/MasterData/WALs/jenkins-hbase4.apache.org,40183,1693151923973 2023-08-27 15:58:44,119 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40183%2C1693151923973, suffix=, logDir=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/MasterData/WALs/jenkins-hbase4.apache.org,40183,1693151923973, archiveDir=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/MasterData/oldWALs, maxLogs=10 2023-08-27 15:58:44,134 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42655,DS-b140b406-079e-4601-935e-db73bafb4d8e,DISK] 2023-08-27 15:58:44,135 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37545,DS-a11783fe-967d-4589-a49d-d8f189794513,DISK] 2023-08-27 15:58:44,138 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/MasterData/WALs/jenkins-hbase4.apache.org,40183,1693151923973/jenkins-hbase4.apache.org%2C40183%2C1693151923973.1693151924119 2023-08-27 15:58:44,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42655,DS-b140b406-079e-4601-935e-db73bafb4d8e,DISK], DatanodeInfoWithStorage[127.0.0.1:37545,DS-a11783fe-967d-4589-a49d-d8f189794513,DISK]] 2023-08-27 15:58:44,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-27 15:58:44,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:44,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:58:44,138 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:58:44,141 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:58:44,143 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-27 15:58:44,144 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-27 15:58:44,145 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:44,146 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:58:44,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:58:44,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:58:44,154 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-27 15:58:44,154 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=861738, jitterRate=0.09575681388378143}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-27 15:58:44,154 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-27 15:58:44,155 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-27 15:58:44,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-27 15:58:44,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-27 15:58:44,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-27 15:58:44,157 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-27 15:58:44,157 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-08-27 15:58:44,157 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-27 15:58:44,158 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-27 15:58:44,159 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-27 15:58:44,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-27 15:58:44,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-27 15:58:44,172 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-27 15:58:44,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-27 15:58:44,173 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-27 15:58:44,175 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:44,176 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-27 15:58:44,177 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-27 15:58:44,178 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-27 15:58:44,180 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-27 15:58:44,180 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-27 15:58:44,180 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:44,181 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,40183,1693151923973, sessionid=0x1004e7746960000, setting cluster-up flag (Was=false) 2023-08-27 15:58:44,185 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:44,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-27 15:58:44,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40183,1693151923973 2023-08-27 15:58:44,197 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:44,203 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-27 15:58:44,204 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40183,1693151923973 2023-08-27 15:58:44,205 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/.hbase-snapshot/.tmp 2023-08-27 15:58:44,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-27 15:58:44,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 15:58:44,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 15:58:44,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 15:58:44,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 15:58:44,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-27 15:58:44,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:44,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-27 15:58:44,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:44,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693151954213 2023-08-27 15:58:44,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-27 15:58:44,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-27 15:58:44,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-27 15:58:44,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-27 15:58:44,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-27 15:58:44,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-27 15:58:44,214 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:44,214 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-27 15:58:44,214 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-27 15:58:44,215 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-27 15:58:44,215 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-27 15:58:44,215 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-27 15:58:44,215 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-27 15:58:44,215 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-27 15:58:44,215 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693151924215,5,FailOnTimeoutGroup] 2023-08-27 15:58:44,215 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693151924215,5,FailOnTimeoutGroup] 2023-08-27 15:58:44,216 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:44,216 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-27 15:58:44,216 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:44,216 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:44,216 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-27 15:58:44,273 INFO [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(951): ClusterId : f914143f-d85e-453f-a298-35d945e11255 2023-08-27 15:58:44,274 DEBUG [RS:0;jenkins-hbase4:38379] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-27 15:58:44,280 DEBUG [RS:0;jenkins-hbase4:38379] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-27 15:58:44,280 DEBUG [RS:0;jenkins-hbase4:38379] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-27 15:58:44,286 DEBUG [RS:0;jenkins-hbase4:38379] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-27 15:58:44,287 DEBUG [RS:0;jenkins-hbase4:38379] zookeeper.ReadOnlyZKClient(139): Connect 0x49cf898d to 127.0.0.1:62874 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-27 15:58:44,291 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-27 15:58:44,293 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-27 15:58:44,293 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18 2023-08-27 15:58:44,294 DEBUG [RS:0;jenkins-hbase4:38379] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47087ec4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-27 15:58:44,294 DEBUG [RS:0;jenkins-hbase4:38379] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4337e2fd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-27 15:58:44,304 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:44,306 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-27 15:58:44,307 DEBUG [RS:0;jenkins-hbase4:38379] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:38379 2023-08-27 15:58:44,307 INFO [RS:0;jenkins-hbase4:38379] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-27 15:58:44,307 INFO [RS:0;jenkins-hbase4:38379] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-27 15:58:44,307 DEBUG [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(1022): About to register with Master. 2023-08-27 15:58:44,308 INFO [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,40183,1693151923973 with isa=jenkins-hbase4.apache.org/172.31.14.131:38379, startcode=1693151924017 2023-08-27 15:58:44,308 DEBUG [RS:0;jenkins-hbase4:38379] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-27 15:58:44,308 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/info 2023-08-27 15:58:44,309 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-27 15:58:44,311 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:44,311 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-27 15:58:44,311 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43381, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-08-27 15:58:44,313 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40183] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:44,313 DEBUG [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18 2023-08-27 15:58:44,314 DEBUG [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:36261 2023-08-27 15:58:44,314 DEBUG [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-27 15:58:44,315 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/rep_barrier 2023-08-27 15:58:44,315 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-27 15:58:44,315 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-27 15:58:44,316 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:44,317 DEBUG [RS:0;jenkins-hbase4:38379] zookeeper.ZKUtil(162): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:44,317 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-27 15:58:44,317 WARN [RS:0;jenkins-hbase4:38379] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-27 15:58:44,317 INFO [RS:0;jenkins-hbase4:38379] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-27 15:58:44,317 DEBUG [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(1948): logDir=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:44,317 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38379,1693151924017] 2023-08-27 15:58:44,319 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/table 2023-08-27 15:58:44,320 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-27 15:58:44,321 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:44,322 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740 2023-08-27 15:58:44,322 DEBUG [RS:0;jenkins-hbase4:38379] zookeeper.ZKUtil(162): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:44,323 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740 2023-08-27 15:58:44,323 DEBUG [RS:0;jenkins-hbase4:38379] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-27 15:58:44,324 INFO [RS:0;jenkins-hbase4:38379] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-27 15:58:44,325 INFO [RS:0;jenkins-hbase4:38379] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-27 15:58:44,325 INFO [RS:0;jenkins-hbase4:38379] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-27 15:58:44,325 INFO [RS:0;jenkins-hbase4:38379] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:44,326 INFO [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-27 15:58:44,326 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-27 15:58:44,327 INFO [RS:0;jenkins-hbase4:38379] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:44,327 DEBUG [RS:0;jenkins-hbase4:38379] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:44,327 DEBUG [RS:0;jenkins-hbase4:38379] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:44,327 DEBUG [RS:0;jenkins-hbase4:38379] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:44,327 DEBUG [RS:0;jenkins-hbase4:38379] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:44,327 DEBUG [RS:0;jenkins-hbase4:38379] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:44,327 DEBUG [RS:0;jenkins-hbase4:38379] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-27 15:58:44,327 DEBUG [RS:0;jenkins-hbase4:38379] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:44,327 DEBUG [RS:0;jenkins-hbase4:38379] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:44,328 DEBUG [RS:0;jenkins-hbase4:38379] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:44,328 DEBUG [RS:0;jenkins-hbase4:38379] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:58:44,328 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-27 15:58:44,328 INFO [RS:0;jenkins-hbase4:38379] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:44,329 INFO [RS:0;jenkins-hbase4:38379] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:44,329 INFO [RS:0;jenkins-hbase4:38379] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:44,331 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-27 15:58:44,331 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=697754, jitterRate=-0.11276032030582428}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-27 15:58:44,332 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-27 15:58:44,332 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-27 15:58:44,332 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-27 15:58:44,332 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-27 15:58:44,332 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-27 15:58:44,332 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-27 15:58:44,332 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-27 15:58:44,332 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-27 15:58:44,334 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-27 15:58:44,334 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-27 15:58:44,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-27 15:58:44,336 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-27 15:58:44,337 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-27 15:58:44,342 INFO [RS:0;jenkins-hbase4:38379] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-27 15:58:44,342 INFO [RS:0;jenkins-hbase4:38379] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38379,1693151924017-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:44,352 INFO [RS:0;jenkins-hbase4:38379] regionserver.Replication(203): jenkins-hbase4.apache.org,38379,1693151924017 started 2023-08-27 15:58:44,352 INFO [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,38379,1693151924017, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38379, sessionid=0x1004e7746960001 2023-08-27 15:58:44,352 DEBUG [RS:0;jenkins-hbase4:38379] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-27 15:58:44,352 DEBUG [RS:0;jenkins-hbase4:38379] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:44,352 DEBUG [RS:0;jenkins-hbase4:38379] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38379,1693151924017' 2023-08-27 15:58:44,352 DEBUG [RS:0;jenkins-hbase4:38379] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-27 15:58:44,353 DEBUG [RS:0;jenkins-hbase4:38379] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-27 15:58:44,353 DEBUG [RS:0;jenkins-hbase4:38379] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-27 15:58:44,353 DEBUG [RS:0;jenkins-hbase4:38379] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-27 15:58:44,353 DEBUG [RS:0;jenkins-hbase4:38379] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:44,353 DEBUG [RS:0;jenkins-hbase4:38379] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38379,1693151924017' 2023-08-27 15:58:44,353 DEBUG [RS:0;jenkins-hbase4:38379] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-27 15:58:44,354 DEBUG [RS:0;jenkins-hbase4:38379] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-27 15:58:44,354 DEBUG [RS:0;jenkins-hbase4:38379] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-27 15:58:44,354 INFO [RS:0;jenkins-hbase4:38379] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-27 15:58:44,354 INFO [RS:0;jenkins-hbase4:38379] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-27 15:58:44,457 INFO [RS:0;jenkins-hbase4:38379] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38379%2C1693151924017, suffix=, logDir=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017, archiveDir=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/oldWALs, maxLogs=32 2023-08-27 15:58:44,479 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37545,DS-a11783fe-967d-4589-a49d-d8f189794513,DISK] 2023-08-27 15:58:44,479 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42655,DS-b140b406-079e-4601-935e-db73bafb4d8e,DISK] 2023-08-27 15:58:44,483 INFO [RS:0;jenkins-hbase4:38379] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017/jenkins-hbase4.apache.org%2C38379%2C1693151924017.1693151924459 2023-08-27 15:58:44,483 DEBUG [RS:0;jenkins-hbase4:38379] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37545,DS-a11783fe-967d-4589-a49d-d8f189794513,DISK], DatanodeInfoWithStorage[127.0.0.1:42655,DS-b140b406-079e-4601-935e-db73bafb4d8e,DISK]] 2023-08-27 15:58:44,488 DEBUG [jenkins-hbase4:40183] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-27 15:58:44,489 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38379,1693151924017, state=OPENING 2023-08-27 15:58:44,490 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-27 15:58:44,491 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:44,492 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-27 15:58:44,492 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38379,1693151924017}] 2023-08-27 15:58:44,647 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:44,648 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-27 15:58:44,650 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44782, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-27 15:58:44,655 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-27 15:58:44,655 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-27 15:58:44,657 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38379%2C1693151924017.meta, suffix=.meta, logDir=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017, archiveDir=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/oldWALs, maxLogs=32 2023-08-27 15:58:44,675 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37545,DS-a11783fe-967d-4589-a49d-d8f189794513,DISK] 2023-08-27 15:58:44,675 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42655,DS-b140b406-079e-4601-935e-db73bafb4d8e,DISK] 2023-08-27 15:58:44,677 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017/jenkins-hbase4.apache.org%2C38379%2C1693151924017.meta.1693151924659.meta 2023-08-27 15:58:44,678 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37545,DS-a11783fe-967d-4589-a49d-d8f189794513,DISK], DatanodeInfoWithStorage[127.0.0.1:42655,DS-b140b406-079e-4601-935e-db73bafb4d8e,DISK]] 2023-08-27 15:58:44,678 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-27 15:58:44,678 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-27 15:58:44,678 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-27 15:58:44,679 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-27 15:58:44,679 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-27 15:58:44,679 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:44,679 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-27 15:58:44,679 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-27 15:58:44,681 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-27 15:58:44,682 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/info 2023-08-27 15:58:44,682 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/info 2023-08-27 15:58:44,682 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-27 15:58:44,683 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:44,683 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-27 15:58:44,684 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/rep_barrier 2023-08-27 15:58:44,684 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/rep_barrier 2023-08-27 15:58:44,685 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-27 15:58:44,685 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:44,685 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-27 15:58:44,687 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/table 2023-08-27 15:58:44,687 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/table 2023-08-27 15:58:44,688 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-27 15:58:44,688 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:44,689 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740 2023-08-27 15:58:44,691 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740 2023-08-27 15:58:44,693 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-27 15:58:44,695 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-27 15:58:44,696 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=851323, jitterRate=0.08251327276229858}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-27 15:58:44,696 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-27 15:58:44,697 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693151924647 2023-08-27 15:58:44,701 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-27 15:58:44,701 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-27 15:58:44,702 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38379,1693151924017, state=OPEN 2023-08-27 15:58:44,706 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-27 15:58:44,706 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-27 15:58:44,709 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-27 15:58:44,709 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38379,1693151924017 in 214 msec 2023-08-27 15:58:44,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-27 15:58:44,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 375 msec 2023-08-27 15:58:44,715 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 507 msec 2023-08-27 15:58:44,715 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693151924715, completionTime=-1 2023-08-27 15:58:44,715 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-27 15:58:44,715 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-27 15:58:44,718 DEBUG [hconnection-0x47f189f4-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-27 15:58:44,720 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44786, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-27 15:58:44,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-27 15:58:44,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693151984721 2023-08-27 15:58:44,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693152044721 2023-08-27 15:58:44,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 6 msec 2023-08-27 15:58:44,727 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40183,1693151923973-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:44,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40183,1693151923973-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:44,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40183,1693151923973-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:44,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:40183, period=300000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:44,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-27 15:58:44,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-27 15:58:44,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-27 15:58:44,729 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-27 15:58:44,729 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-27 15:58:44,731 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-27 15:58:44,732 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-27 15:58:44,734 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/.tmp/data/hbase/namespace/12a0d634325e65d9423a1f5e9f94adb7 2023-08-27 15:58:44,734 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/.tmp/data/hbase/namespace/12a0d634325e65d9423a1f5e9f94adb7 empty. 2023-08-27 15:58:44,735 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/.tmp/data/hbase/namespace/12a0d634325e65d9423a1f5e9f94adb7 2023-08-27 15:58:44,735 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-27 15:58:44,750 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-27 15:58:44,751 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 12a0d634325e65d9423a1f5e9f94adb7, NAME => 'hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/.tmp 2023-08-27 15:58:44,761 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:44,761 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 12a0d634325e65d9423a1f5e9f94adb7, disabling compactions & flushes 2023-08-27 15:58:44,761 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. 2023-08-27 15:58:44,761 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. 2023-08-27 15:58:44,762 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. after waiting 0 ms 2023-08-27 15:58:44,762 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. 2023-08-27 15:58:44,762 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. 2023-08-27 15:58:44,762 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 12a0d634325e65d9423a1f5e9f94adb7: 2023-08-27 15:58:44,764 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-27 15:58:44,766 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693151924766"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693151924766"}]},"ts":"1693151924766"} 2023-08-27 15:58:44,768 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-27 15:58:44,769 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-27 15:58:44,769 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693151924769"}]},"ts":"1693151924769"} 2023-08-27 15:58:44,774 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-27 15:58:44,780 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=12a0d634325e65d9423a1f5e9f94adb7, ASSIGN}] 2023-08-27 15:58:44,782 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=12a0d634325e65d9423a1f5e9f94adb7, ASSIGN 2023-08-27 15:58:44,783 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=12a0d634325e65d9423a1f5e9f94adb7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38379,1693151924017; forceNewPlan=false, retain=false 2023-08-27 15:58:44,834 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-27 15:58:44,934 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=12a0d634325e65d9423a1f5e9f94adb7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:44,935 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693151924934"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693151924934"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693151924934"}]},"ts":"1693151924934"} 2023-08-27 15:58:44,938 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 12a0d634325e65d9423a1f5e9f94adb7, server=jenkins-hbase4.apache.org,38379,1693151924017}] 2023-08-27 15:58:45,098 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. 2023-08-27 15:58:45,098 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 12a0d634325e65d9423a1f5e9f94adb7, NAME => 'hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7.', STARTKEY => '', ENDKEY => ''} 2023-08-27 15:58:45,099 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 12a0d634325e65d9423a1f5e9f94adb7 2023-08-27 15:58:45,099 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:45,099 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 12a0d634325e65d9423a1f5e9f94adb7 2023-08-27 15:58:45,099 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 12a0d634325e65d9423a1f5e9f94adb7 2023-08-27 15:58:45,103 INFO [StoreOpener-12a0d634325e65d9423a1f5e9f94adb7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 12a0d634325e65d9423a1f5e9f94adb7 2023-08-27 15:58:45,105 DEBUG [StoreOpener-12a0d634325e65d9423a1f5e9f94adb7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/namespace/12a0d634325e65d9423a1f5e9f94adb7/info 2023-08-27 15:58:45,105 DEBUG [StoreOpener-12a0d634325e65d9423a1f5e9f94adb7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/namespace/12a0d634325e65d9423a1f5e9f94adb7/info 2023-08-27 15:58:45,105 INFO [StoreOpener-12a0d634325e65d9423a1f5e9f94adb7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 12a0d634325e65d9423a1f5e9f94adb7 columnFamilyName info 2023-08-27 15:58:45,106 INFO [StoreOpener-12a0d634325e65d9423a1f5e9f94adb7-1] regionserver.HStore(310): Store=12a0d634325e65d9423a1f5e9f94adb7/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:45,107 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/namespace/12a0d634325e65d9423a1f5e9f94adb7 2023-08-27 15:58:45,108 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/namespace/12a0d634325e65d9423a1f5e9f94adb7 2023-08-27 15:58:45,112 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 12a0d634325e65d9423a1f5e9f94adb7 2023-08-27 15:58:45,115 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/namespace/12a0d634325e65d9423a1f5e9f94adb7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-27 15:58:45,116 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 12a0d634325e65d9423a1f5e9f94adb7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=831023, jitterRate=0.05670081079006195}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-27 15:58:45,116 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 12a0d634325e65d9423a1f5e9f94adb7: 2023-08-27 15:58:45,117 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7., pid=6, masterSystemTime=1693151925091 2023-08-27 15:58:45,121 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=12a0d634325e65d9423a1f5e9f94adb7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:45,122 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693151925121"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693151925121"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693151925121"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693151925121"}]},"ts":"1693151925121"} 2023-08-27 15:58:45,122 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. 2023-08-27 15:58:45,122 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. 2023-08-27 15:58:45,127 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-27 15:58:45,128 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 12a0d634325e65d9423a1f5e9f94adb7, server=jenkins-hbase4.apache.org,38379,1693151924017 in 187 msec 2023-08-27 15:58:45,131 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-27 15:58:45,131 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=12a0d634325e65d9423a1f5e9f94adb7, ASSIGN in 348 msec 2023-08-27 15:58:45,132 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-27 15:58:45,132 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693151925132"}]},"ts":"1693151925132"} 2023-08-27 15:58:45,133 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-27 15:58:45,136 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-27 15:58:45,138 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 409 msec 2023-08-27 15:58:45,231 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-27 15:58:45,232 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-27 15:58:45,232 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:45,239 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-27 15:58:45,251 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-27 15:58:45,255 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 15 msec 2023-08-27 15:58:45,262 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-27 15:58:45,271 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-27 15:58:45,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 13 msec 2023-08-27 15:58:45,295 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-27 15:58:45,297 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-27 15:58:45,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.253sec 2023-08-27 15:58:45,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-27 15:58:45,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-27 15:58:45,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-27 15:58:45,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40183,1693151923973-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-27 15:58:45,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40183,1693151923973-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-27 15:58:45,301 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-27 15:58:45,373 DEBUG [Listener at localhost/40339] zookeeper.ReadOnlyZKClient(139): Connect 0x0ed62c52 to 127.0.0.1:62874 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-27 15:58:45,379 DEBUG [Listener at localhost/40339] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2b34092c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-27 15:58:45,381 DEBUG [hconnection-0x4e00d43-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-27 15:58:45,383 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42296, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-27 15:58:45,385 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,40183,1693151923973 2023-08-27 15:58:45,386 INFO [Listener at localhost/40339] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:58:45,390 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-27 15:58:45,390 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:58:45,391 INFO [Listener at localhost/40339] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-27 15:58:45,392 DEBUG [Listener at localhost/40339] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-27 15:58:45,398 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36448, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-27 15:58:45,400 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-08-27 15:58:45,400 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-08-27 15:58:45,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-27 15:58:45,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:58:45,404 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-08-27 15:58:45,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testCompactionRecordDoesntBlockRolling" procId is: 9 2023-08-27 15:58:45,405 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-27 15:58:45,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-27 15:58:45,408 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051 2023-08-27 15:58:45,408 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051 empty. 2023-08-27 15:58:45,409 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051 2023-08-27 15:58:45,409 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testCompactionRecordDoesntBlockRolling regions 2023-08-27 15:58:45,429 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/.tabledesc/.tableinfo.0000000001 2023-08-27 15:58:45,431 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1da8c0df0cf8bfdcb0dcf4ea915d7051, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/.tmp 2023-08-27 15:58:45,443 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:45,443 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1604): Closing 1da8c0df0cf8bfdcb0dcf4ea915d7051, disabling compactions & flushes 2023-08-27 15:58:45,443 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:58:45,443 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:58:45,443 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. after waiting 0 ms 2023-08-27 15:58:45,443 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:58:45,443 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:58:45,444 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1558): Region close journal for 1da8c0df0cf8bfdcb0dcf4ea915d7051: 2023-08-27 15:58:45,447 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ADD_TO_META 2023-08-27 15:58:45,448 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1693151925448"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693151925448"}]},"ts":"1693151925448"} 2023-08-27 15:58:45,451 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-27 15:58:45,452 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-27 15:58:45,452 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693151925452"}]},"ts":"1693151925452"} 2023-08-27 15:58:45,454 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLING in hbase:meta 2023-08-27 15:58:45,459 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=1da8c0df0cf8bfdcb0dcf4ea915d7051, ASSIGN}] 2023-08-27 15:58:45,461 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=1da8c0df0cf8bfdcb0dcf4ea915d7051, ASSIGN 2023-08-27 15:58:45,462 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=1da8c0df0cf8bfdcb0dcf4ea915d7051, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38379,1693151924017; forceNewPlan=false, retain=false 2023-08-27 15:58:45,613 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=1da8c0df0cf8bfdcb0dcf4ea915d7051, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:45,614 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1693151925613"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693151925613"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693151925613"}]},"ts":"1693151925613"} 2023-08-27 15:58:45,616 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 1da8c0df0cf8bfdcb0dcf4ea915d7051, server=jenkins-hbase4.apache.org,38379,1693151924017}] 2023-08-27 15:58:45,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:58:45,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1da8c0df0cf8bfdcb0dcf4ea915d7051, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051.', STARTKEY => '', ENDKEY => ''} 2023-08-27 15:58:45,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testCompactionRecordDoesntBlockRolling 1da8c0df0cf8bfdcb0dcf4ea915d7051 2023-08-27 15:58:45,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:58:45,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1da8c0df0cf8bfdcb0dcf4ea915d7051 2023-08-27 15:58:45,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1da8c0df0cf8bfdcb0dcf4ea915d7051 2023-08-27 15:58:45,777 INFO [StoreOpener-1da8c0df0cf8bfdcb0dcf4ea915d7051-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1da8c0df0cf8bfdcb0dcf4ea915d7051 2023-08-27 15:58:45,779 DEBUG [StoreOpener-1da8c0df0cf8bfdcb0dcf4ea915d7051-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info 2023-08-27 15:58:45,779 DEBUG [StoreOpener-1da8c0df0cf8bfdcb0dcf4ea915d7051-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info 2023-08-27 15:58:45,780 INFO [StoreOpener-1da8c0df0cf8bfdcb0dcf4ea915d7051-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1da8c0df0cf8bfdcb0dcf4ea915d7051 columnFamilyName info 2023-08-27 15:58:45,781 INFO [StoreOpener-1da8c0df0cf8bfdcb0dcf4ea915d7051-1] regionserver.HStore(310): Store=1da8c0df0cf8bfdcb0dcf4ea915d7051/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:58:45,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051 2023-08-27 15:58:45,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051 2023-08-27 15:58:45,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1da8c0df0cf8bfdcb0dcf4ea915d7051 2023-08-27 15:58:45,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-27 15:58:45,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1da8c0df0cf8bfdcb0dcf4ea915d7051; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=838870, jitterRate=0.06667867302894592}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-27 15:58:45,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1da8c0df0cf8bfdcb0dcf4ea915d7051: 2023-08-27 15:58:45,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051., pid=11, masterSystemTime=1693151925769 2023-08-27 15:58:45,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:58:45,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:58:45,793 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=1da8c0df0cf8bfdcb0dcf4ea915d7051, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:45,793 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1693151925793"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693151925793"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693151925793"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693151925793"}]},"ts":"1693151925793"} 2023-08-27 15:58:45,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-27 15:58:45,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 1da8c0df0cf8bfdcb0dcf4ea915d7051, server=jenkins-hbase4.apache.org,38379,1693151924017 in 179 msec 2023-08-27 15:58:45,800 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-27 15:58:45,801 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=1da8c0df0cf8bfdcb0dcf4ea915d7051, ASSIGN in 339 msec 2023-08-27 15:58:45,802 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-27 15:58:45,802 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693151925802"}]},"ts":"1693151925802"} 2023-08-27 15:58:45,803 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLED in hbase:meta 2023-08-27 15:58:45,806 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_POST_OPERATION 2023-08-27 15:58:45,808 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling in 405 msec 2023-08-27 15:58:49,331 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-27 15:58:49,332 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-08-27 15:58:50,182 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-27 15:58:50,324 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-27 15:58:50,324 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-27 15:58:50,325 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-27 15:58:55,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-27 15:58:55,408 INFO [Listener at localhost/40339] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testCompactionRecordDoesntBlockRolling, procId: 9 completed 2023-08-27 15:58:55,411 DEBUG [Listener at localhost/40339] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:58:55,411 DEBUG [Listener at localhost/40339] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:58:55,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-27 15:58:55,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] procedure.ProcedureCoordinator(165): Submitting procedure hbase:namespace 2023-08-27 15:58:55,430 INFO [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'hbase:namespace' 2023-08-27 15:58:55,430 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-27 15:58:55,430 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'hbase:namespace' starting 'acquire' 2023-08-27 15:58:55,430 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'hbase:namespace', kicking off acquire phase on members. 2023-08-27 15:58:55,431 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-08-27 15:58:55,431 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-27 15:58:55,432 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-27 15:58:55,432 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-27 15:58:55,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-27 15:58:55,433 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,433 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-27 15:58:55,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/hbase:namespace 2023-08-27 15:58:55,433 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-08-27 15:58:55,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-27 15:58:55,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-27 15:58:55,434 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for hbase:namespace 2023-08-27 15:58:55,436 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:hbase:namespace 2023-08-27 15:58:55,436 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'hbase:namespace' with timeout 60000ms 2023-08-27 15:58:55,436 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-27 15:58:55,437 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'hbase:namespace' starting 'acquire' stage 2023-08-27 15:58:55,437 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-27 15:58:55,437 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-27 15:58:55,437 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. 2023-08-27 15:58:55,437 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. started... 2023-08-27 15:58:55,438 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 12a0d634325e65d9423a1f5e9f94adb7 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-27 15:58:55,450 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/namespace/12a0d634325e65d9423a1f5e9f94adb7/.tmp/info/d435d6d4a01d4568a2acbef65155b854 2023-08-27 15:58:55,460 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/namespace/12a0d634325e65d9423a1f5e9f94adb7/.tmp/info/d435d6d4a01d4568a2acbef65155b854 as hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/namespace/12a0d634325e65d9423a1f5e9f94adb7/info/d435d6d4a01d4568a2acbef65155b854 2023-08-27 15:58:55,467 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/namespace/12a0d634325e65d9423a1f5e9f94adb7/info/d435d6d4a01d4568a2acbef65155b854, entries=2, sequenceid=6, filesize=4.8 K 2023-08-27 15:58:55,469 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 12a0d634325e65d9423a1f5e9f94adb7 in 31ms, sequenceid=6, compaction requested=false 2023-08-27 15:58:55,469 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 12a0d634325e65d9423a1f5e9f94adb7: 2023-08-27 15:58:55,469 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. 2023-08-27 15:58:55,469 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-27 15:58:55,469 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-27 15:58:55,469 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,469 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'hbase:namespace' locally acquired 2023-08-27 15:58:55,469 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38379,1693151924017' joining acquired barrier for procedure (hbase:namespace) in zk 2023-08-27 15:58:55,471 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,471 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-27 15:58:55,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-27 15:58:55,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-27 15:58:55,472 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace 2023-08-27 15:58:55,472 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'hbase:namespace' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-27 15:58:55,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-27 15:58:55,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-27 15:58:55,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-27 15:58:55,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-27 15:58:55,473 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38379,1693151924017' joining acquired barrier for procedure 'hbase:namespace' on coordinator 2023-08-27 15:58:55,474 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'hbase:namespace' starting 'in-barrier' execution. 2023-08-27 15:58:55,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9fb67bd[Count = 0] remaining members to acquire global barrier 2023-08-27 15:58:55,474 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-27 15:58:55,476 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-08-27 15:58:55,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-27 15:58:55,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-27 15:58:55,476 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'hbase:namespace' received 'reached' from coordinator. 2023-08-27 15:58:55,476 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'hbase:namespace' locally completed 2023-08-27 15:58:55,476 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'hbase:namespace' completed for member 'jenkins-hbase4.apache.org,38379,1693151924017' in zk 2023-08-27 15:58:55,476 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,476 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-27 15:58:55,478 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,478 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'hbase:namespace' has notified controller of completion 2023-08-27 15:58:55,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-27 15:58:55,478 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-27 15:58:55,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-27 15:58:55,479 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'hbase:namespace' completed. 2023-08-27 15:58:55,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-27 15:58:55,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-27 15:58:55,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-27 15:58:55,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-27 15:58:55,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-27 15:58:55,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'hbase:namespace' member 'jenkins-hbase4.apache.org,38379,1693151924017': 2023-08-27 15:58:55,482 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38379,1693151924017' released barrier for procedure'hbase:namespace', counting down latch. Waiting for 0 more 2023-08-27 15:58:55,482 INFO [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'hbase:namespace' execution completed 2023-08-27 15:58:55,482 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-27 15:58:55,482 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-27 15:58:55,482 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:hbase:namespace 2023-08-27 15:58:55,482 INFO [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure hbase:namespaceincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-27 15:58:55,484 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-08-27 15:58:55,484 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-08-27 15:58:55,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/hbase:namespace 2023-08-27 15:58:55,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/hbase:namespace 2023-08-27 15:58:55,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-27 15:58:55,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-08-27 15:58:55,484 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-27 15:58:55,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-27 15:58:55,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-27 15:58:55,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-27 15:58:55,484 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-27 15:58:55,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-08-27 15:58:55,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-27 15:58:55,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-27 15:58:55,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-27 15:58:55,486 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-27 15:58:55,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-27 15:58:55,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,492 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,492 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-27 15:58:55,492 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-27 15:58:55,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-27 15:58:55,493 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-27 15:58:55,492 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-27 15:58:55,493 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:58:55,493 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'hbase:namespace' 2023-08-27 15:58:55,493 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-27 15:58:55,493 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-27 15:58:55,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-27 15:58:55,493 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-08-27 15:58:55,493 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-08-27 15:58:55,493 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-08-27 15:58:55,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-27 15:58:55,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-27 15:58:55,495 DEBUG [Listener at localhost/40339] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : hbase:namespace'' to complete. (max 20000 ms per retry) 2023-08-27 15:58:55,495 DEBUG [Listener at localhost/40339] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-27 15:59:05,495 DEBUG [Listener at localhost/40339] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-27 15:59:05,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-27 15:59:05,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-27 15:59:05,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,530 INFO [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-27 15:59:05,530 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-27 15:59:05,531 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-27 15:59:05,531 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-27 15:59:05,532 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,532 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,534 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,534 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-27 15:59:05,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-27 15:59:05,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-27 15:59:05,534 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,534 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-27 15:59:05,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,535 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-27 15:59:05,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,536 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,536 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,536 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-27 15:59:05,536 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-27 15:59:05,537 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-27 15:59:05,537 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-27 15:59:05,537 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-27 15:59:05,537 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:59:05,537 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. started... 2023-08-27 15:59:05,538 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1da8c0df0cf8bfdcb0dcf4ea915d7051 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-27 15:59:05,555 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=5 (bloomFilter=true), to=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/.tmp/info/2fc40b1c6e094b2e9698174ae746c101 2023-08-27 15:59:05,565 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/.tmp/info/2fc40b1c6e094b2e9698174ae746c101 as hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/2fc40b1c6e094b2e9698174ae746c101 2023-08-27 15:59:05,572 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/2fc40b1c6e094b2e9698174ae746c101, entries=1, sequenceid=5, filesize=5.8 K 2023-08-27 15:59:05,574 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 1da8c0df0cf8bfdcb0dcf4ea915d7051 in 36ms, sequenceid=5, compaction requested=false 2023-08-27 15:59:05,574 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1da8c0df0cf8bfdcb0dcf4ea915d7051: 2023-08-27 15:59:05,574 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:59:05,574 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-27 15:59:05,575 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-27 15:59:05,575 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,575 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-27 15:59:05,575 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38379,1693151924017' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-27 15:59:05,577 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,577 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-27 15:59:05,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-27 15:59:05,577 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,577 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-27 15:59:05,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-27 15:59:05,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-27 15:59:05,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-27 15:59:05,580 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38379,1693151924017' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-27 15:59:05,580 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@dc4e25a[Count = 0] remaining members to acquire global barrier 2023-08-27 15:59:05,580 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-27 15:59:05,580 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,583 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,583 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-27 15:59:05,583 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-27 15:59:05,583 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,38379,1693151924017' in zk 2023-08-27 15:59:05,583 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,583 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-27 15:59:05,585 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-27 15:59:05,585 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,585 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-27 15:59:05,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-27 15:59:05,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-27 15:59:05,585 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-27 15:59:05,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-27 15:59:05,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-27 15:59:05,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-27 15:59:05,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,38379,1693151924017': 2023-08-27 15:59:05,589 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38379,1693151924017' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-27 15:59:05,589 INFO [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-27 15:59:05,589 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-27 15:59:05,589 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-27 15:59:05,589 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,589 INFO [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-27 15:59:05,595 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,595 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-27 15:59:05,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-27 15:59:05,595 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-27 15:59:05,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-27 15:59:05,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-27 15:59:05,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-27 15:59:05,596 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-27 15:59:05,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,598 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-27 15:59:05,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,600 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,600 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-27 15:59:05,600 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-27 15:59:05,601 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-27 15:59:05,601 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-27 15:59:05,601 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-27 15:59:05,601 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-08-27 15:59:05,601 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:05,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-27 15:59:05,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-27 15:59:05,601 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-27 15:59:05,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-27 15:59:05,602 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,602 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:05,602 DEBUG [Listener at localhost/40339] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-27 15:59:05,602 DEBUG [Listener at localhost/40339] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-27 15:59:15,602 DEBUG [Listener at localhost/40339] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-27 15:59:15,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-27 15:59:15,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-27 15:59:15,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-08-27 15:59:15,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,616 INFO [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-27 15:59:15,616 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-27 15:59:15,617 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-27 15:59:15,617 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-27 15:59:15,618 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,618 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,619 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-27 15:59:15,619 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-27 15:59:15,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-27 15:59:15,620 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,620 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-27 15:59:15,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,620 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-27 15:59:15,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,620 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,621 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-08-27 15:59:15,621 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,621 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-27 15:59:15,621 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-27 15:59:15,621 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-27 15:59:15,622 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-27 15:59:15,622 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-27 15:59:15,622 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:59:15,622 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. started... 2023-08-27 15:59:15,622 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1da8c0df0cf8bfdcb0dcf4ea915d7051 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-27 15:59:15,644 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=9 (bloomFilter=true), to=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/.tmp/info/59a2d566bf3a4b96a18d278bd7a9d9b8 2023-08-27 15:59:15,654 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/.tmp/info/59a2d566bf3a4b96a18d278bd7a9d9b8 as hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/59a2d566bf3a4b96a18d278bd7a9d9b8 2023-08-27 15:59:15,661 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/59a2d566bf3a4b96a18d278bd7a9d9b8, entries=1, sequenceid=9, filesize=5.8 K 2023-08-27 15:59:15,662 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 1da8c0df0cf8bfdcb0dcf4ea915d7051 in 40ms, sequenceid=9, compaction requested=false 2023-08-27 15:59:15,662 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1da8c0df0cf8bfdcb0dcf4ea915d7051: 2023-08-27 15:59:15,662 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:59:15,662 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-27 15:59:15,663 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-27 15:59:15,663 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,663 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-27 15:59:15,663 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38379,1693151924017' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-27 15:59:15,665 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,665 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-27 15:59:15,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-27 15:59:15,666 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,666 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-27 15:59:15,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-27 15:59:15,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-27 15:59:15,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-27 15:59:15,667 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38379,1693151924017' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-27 15:59:15,667 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48c66d86[Count = 0] remaining members to acquire global barrier 2023-08-27 15:59:15,667 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-27 15:59:15,667 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,669 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,669 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-27 15:59:15,669 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-27 15:59:15,669 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,38379,1693151924017' in zk 2023-08-27 15:59:15,673 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,673 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-27 15:59:15,675 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-27 15:59:15,675 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,675 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-27 15:59:15,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-27 15:59:15,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-27 15:59:15,675 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-27 15:59:15,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-27 15:59:15,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-27 15:59:15,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-27 15:59:15,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,38379,1693151924017': 2023-08-27 15:59:15,678 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38379,1693151924017' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-27 15:59:15,678 INFO [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-27 15:59:15,678 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-27 15:59:15,678 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-27 15:59:15,678 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,678 INFO [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-27 15:59:15,680 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,680 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,680 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-27 15:59:15,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-27 15:59:15,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-27 15:59:15,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-27 15:59:15,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-27 15:59:15,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-27 15:59:15,681 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-27 15:59:15,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,682 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-27 15:59:15,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,685 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,685 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,685 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-27 15:59:15,685 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,685 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-08-27 15:59:15,685 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:15,685 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-27 15:59:15,685 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-27 15:59:15,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-27 15:59:15,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-27 15:59:15,685 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-27 15:59:15,685 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-27 15:59:15,685 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,686 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,686 DEBUG [Listener at localhost/40339] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-27 15:59:15,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-27 15:59:15,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-27 15:59:15,686 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:15,686 DEBUG [Listener at localhost/40339] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-27 15:59:25,686 DEBUG [Listener at localhost/40339] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-27 15:59:25,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-27 15:59:25,707 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42655,DS-b140b406-079e-4601-935e-db73bafb4d8e,DISK] 2023-08-27 15:59:25,707 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37545,DS-a11783fe-967d-4589-a49d-d8f189794513,DISK] 2023-08-27 15:59:25,711 INFO [Listener at localhost/40339] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017/jenkins-hbase4.apache.org%2C38379%2C1693151924017.1693151924459 with entries=13, filesize=6.36 KB; new WAL /user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017/jenkins-hbase4.apache.org%2C38379%2C1693151924017.1693151965691 2023-08-27 15:59:25,711 DEBUG [Listener at localhost/40339] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42655,DS-b140b406-079e-4601-935e-db73bafb4d8e,DISK], DatanodeInfoWithStorage[127.0.0.1:37545,DS-a11783fe-967d-4589-a49d-d8f189794513,DISK]] 2023-08-27 15:59:25,711 DEBUG [Listener at localhost/40339] wal.AbstractFSWAL(716): hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017/jenkins-hbase4.apache.org%2C38379%2C1693151924017.1693151924459 is not closed yet, will try archiving it next time 2023-08-27 15:59:25,714 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017/jenkins-hbase4.apache.org%2C38379%2C1693151924017.1693151924459 not finished, retry = 0 2023-08-27 15:59:25,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-27 15:59:25,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-08-27 15:59:25,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,718 INFO [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-27 15:59:25,718 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-27 15:59:25,719 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-27 15:59:25,719 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-27 15:59:25,719 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,719 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,721 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-27 15:59:25,721 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-27 15:59:25,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-27 15:59:25,721 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,721 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-27 15:59:25,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,722 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-27 15:59:25,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,722 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,722 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-08-27 15:59:25,722 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,722 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-27 15:59:25,722 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-27 15:59:25,723 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-27 15:59:25,723 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-27 15:59:25,723 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-27 15:59:25,723 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:59:25,723 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. started... 2023-08-27 15:59:25,723 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1da8c0df0cf8bfdcb0dcf4ea915d7051 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-27 15:59:25,734 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=13 (bloomFilter=true), to=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/.tmp/info/3812ee15e2894aaeadcb9aaa02ee8b73 2023-08-27 15:59:25,741 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/.tmp/info/3812ee15e2894aaeadcb9aaa02ee8b73 as hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/3812ee15e2894aaeadcb9aaa02ee8b73 2023-08-27 15:59:25,747 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/3812ee15e2894aaeadcb9aaa02ee8b73, entries=1, sequenceid=13, filesize=5.8 K 2023-08-27 15:59:25,749 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 1da8c0df0cf8bfdcb0dcf4ea915d7051 in 25ms, sequenceid=13, compaction requested=true 2023-08-27 15:59:25,749 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1da8c0df0cf8bfdcb0dcf4ea915d7051: 2023-08-27 15:59:25,749 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:59:25,749 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-27 15:59:25,749 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-27 15:59:25,749 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,749 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-27 15:59:25,749 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38379,1693151924017' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-27 15:59:25,751 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,751 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-27 15:59:25,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-27 15:59:25,751 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,751 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-27 15:59:25,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-27 15:59:25,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-27 15:59:25,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-27 15:59:25,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38379,1693151924017' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-27 15:59:25,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2bd0c86d[Count = 0] remaining members to acquire global barrier 2023-08-27 15:59:25,753 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-27 15:59:25,753 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,754 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,754 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-27 15:59:25,754 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-27 15:59:25,754 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,38379,1693151924017' in zk 2023-08-27 15:59:25,754 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,754 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-27 15:59:25,756 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-27 15:59:25,756 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,756 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-27 15:59:25,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-27 15:59:25,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-27 15:59:25,756 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-27 15:59:25,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-27 15:59:25,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-27 15:59:25,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-27 15:59:25,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,38379,1693151924017': 2023-08-27 15:59:25,759 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38379,1693151924017' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-27 15:59:25,759 INFO [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-27 15:59:25,759 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-27 15:59:25,759 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-27 15:59:25,759 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,759 INFO [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-27 15:59:25,761 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,761 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-27 15:59:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-27 15:59:25,761 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-27 15:59:25,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-27 15:59:25,762 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-27 15:59:25,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-27 15:59:25,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-27 15:59:25,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,763 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-27 15:59:25,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,765 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,765 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,765 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-27 15:59:25,765 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-27 15:59:25,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-27 15:59:25,765 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-08-27 15:59:25,765 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:25,765 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-27 15:59:25,766 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-27 15:59:25,765 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-27 15:59:25,765 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-27 15:59:25,766 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-27 15:59:25,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-27 15:59:25,766 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,766 DEBUG [Listener at localhost/40339] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-27 15:59:25,766 DEBUG [Listener at localhost/40339] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-27 15:59:25,766 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:25,815 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017/jenkins-hbase4.apache.org%2C38379%2C1693151924017.1693151924459 to hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/oldWALs/jenkins-hbase4.apache.org%2C38379%2C1693151924017.1693151924459 2023-08-27 15:59:35,767 DEBUG [Listener at localhost/40339] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-27 15:59:35,768 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-27 15:59:35,769 DEBUG [Listener at localhost/40339] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-27 15:59:35,773 DEBUG [Listener at localhost/40339] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 17694 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-27 15:59:35,775 DEBUG [Listener at localhost/40339] regionserver.HStore(1912): 1da8c0df0cf8bfdcb0dcf4ea915d7051/info is initiating minor compaction (all files) 2023-08-27 15:59:35,775 INFO [Listener at localhost/40339] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-27 15:59:35,775 INFO [Listener at localhost/40339] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:35,776 INFO [Listener at localhost/40339] regionserver.HRegion(2259): Starting compaction of 1da8c0df0cf8bfdcb0dcf4ea915d7051/info in TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:59:35,776 INFO [Listener at localhost/40339] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/2fc40b1c6e094b2e9698174ae746c101, hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/59a2d566bf3a4b96a18d278bd7a9d9b8, hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/3812ee15e2894aaeadcb9aaa02ee8b73] into tmpdir=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/.tmp, totalSize=17.3 K 2023-08-27 15:59:35,777 DEBUG [Listener at localhost/40339] compactions.Compactor(207): Compacting 2fc40b1c6e094b2e9698174ae746c101, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1693151945519 2023-08-27 15:59:35,777 DEBUG [Listener at localhost/40339] compactions.Compactor(207): Compacting 59a2d566bf3a4b96a18d278bd7a9d9b8, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1693151955605 2023-08-27 15:59:35,778 DEBUG [Listener at localhost/40339] compactions.Compactor(207): Compacting 3812ee15e2894aaeadcb9aaa02ee8b73, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1693151965689 2023-08-27 15:59:35,800 INFO [Listener at localhost/40339] throttle.PressureAwareThroughputController(145): 1da8c0df0cf8bfdcb0dcf4ea915d7051#info#compaction#9 average throughput is 3.01 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-27 15:59:35,816 DEBUG [Listener at localhost/40339] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/.tmp/info/8c38f8fc60a947799ce0beef8623b3be as hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/8c38f8fc60a947799ce0beef8623b3be 2023-08-27 15:59:35,830 INFO [Listener at localhost/40339] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1da8c0df0cf8bfdcb0dcf4ea915d7051/info of 1da8c0df0cf8bfdcb0dcf4ea915d7051 into 8c38f8fc60a947799ce0beef8623b3be(size=7.9 K), total size for store is 7.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-27 15:59:35,830 DEBUG [Listener at localhost/40339] regionserver.HRegion(2289): Compaction status journal for 1da8c0df0cf8bfdcb0dcf4ea915d7051: 2023-08-27 15:59:35,846 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37545,DS-a11783fe-967d-4589-a49d-d8f189794513,DISK] 2023-08-27 15:59:35,847 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42655,DS-b140b406-079e-4601-935e-db73bafb4d8e,DISK] 2023-08-27 15:59:35,850 INFO [Listener at localhost/40339] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017/jenkins-hbase4.apache.org%2C38379%2C1693151924017.1693151965691 with entries=4, filesize=2.43 KB; new WAL /user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017/jenkins-hbase4.apache.org%2C38379%2C1693151924017.1693151975832 2023-08-27 15:59:35,850 DEBUG [Listener at localhost/40339] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37545,DS-a11783fe-967d-4589-a49d-d8f189794513,DISK], DatanodeInfoWithStorage[127.0.0.1:42655,DS-b140b406-079e-4601-935e-db73bafb4d8e,DISK]] 2023-08-27 15:59:35,850 DEBUG [Listener at localhost/40339] wal.AbstractFSWAL(716): hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017/jenkins-hbase4.apache.org%2C38379%2C1693151924017.1693151965691 is not closed yet, will try archiving it next time 2023-08-27 15:59:35,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-27 15:59:35,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-08-27 15:59:35,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,858 INFO [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-27 15:59:35,858 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-27 15:59:35,858 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-27 15:59:35,858 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-27 15:59:35,859 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,859 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,869 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,869 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-27 15:59:35,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-27 15:59:35,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-27 15:59:35,869 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,869 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-27 15:59:35,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,870 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-27 15:59:35,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,870 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,870 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-08-27 15:59:35,870 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,870 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-27 15:59:35,870 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-27 15:59:35,871 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-27 15:59:35,871 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-27 15:59:35,871 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-27 15:59:35,871 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:59:35,871 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. started... 2023-08-27 15:59:35,871 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1da8c0df0cf8bfdcb0dcf4ea915d7051 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-27 15:59:35,887 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=18 (bloomFilter=true), to=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/.tmp/info/bd38907391434551af09b153f16b0d5d 2023-08-27 15:59:35,895 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/.tmp/info/bd38907391434551af09b153f16b0d5d as hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/bd38907391434551af09b153f16b0d5d 2023-08-27 15:59:35,902 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/bd38907391434551af09b153f16b0d5d, entries=1, sequenceid=18, filesize=5.8 K 2023-08-27 15:59:35,903 INFO [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 1da8c0df0cf8bfdcb0dcf4ea915d7051 in 32ms, sequenceid=18, compaction requested=false 2023-08-27 15:59:35,903 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1da8c0df0cf8bfdcb0dcf4ea915d7051: 2023-08-27 15:59:35,903 DEBUG [rs(jenkins-hbase4.apache.org,38379,1693151924017)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:59:35,903 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-27 15:59:35,904 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-27 15:59:35,904 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,904 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-27 15:59:35,904 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38379,1693151924017' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-27 15:59:35,906 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,906 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-27 15:59:35,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-27 15:59:35,906 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,906 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-27 15:59:35,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-27 15:59:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-27 15:59:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-27 15:59:35,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38379,1693151924017' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-27 15:59:35,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79c48425[Count = 0] remaining members to acquire global barrier 2023-08-27 15:59:35,908 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-27 15:59:35,908 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,909 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,909 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-27 15:59:35,909 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-27 15:59:35,909 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,910 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-27 15:59:35,909 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,38379,1693151924017' in zk 2023-08-27 15:59:35,912 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,913 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-27 15:59:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,913 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-27 15:59:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-27 15:59:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-27 15:59:35,913 DEBUG [member: 'jenkins-hbase4.apache.org,38379,1693151924017' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-27 15:59:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-27 15:59:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-27 15:59:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-27 15:59:35,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,38379,1693151924017': 2023-08-27 15:59:35,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38379,1693151924017' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-27 15:59:35,916 INFO [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-27 15:59:35,916 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-27 15:59:35,916 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-27 15:59:35,916 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,916 INFO [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-27 15:59:35,918 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,918 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-27 15:59:35,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-27 15:59:35,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,918 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-27 15:59:35,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-27 15:59:35,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-27 15:59:35,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-27 15:59:35,919 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-27 15:59:35,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,921 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-27 15:59:35,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,923 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,923 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-27 15:59:35,923 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-27 15:59:35,923 DEBUG [(jenkins-hbase4.apache.org,40183,1693151923973)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-27 15:59:35,923 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-27 15:59:35,923 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-08-27 15:59:35,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-27 15:59:35,923 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-27 15:59:35,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-27 15:59:35,924 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:35,924 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,924 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,924 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-27 15:59:35,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-27 15:59:35,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-27 15:59:35,925 DEBUG [Listener at localhost/40339] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-27 15:59:35,925 DEBUG [Listener at localhost/40339] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-27 15:59:45,925 DEBUG [Listener at localhost/40339] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-27 15:59:45,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40183] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-27 15:59:45,944 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42655,DS-b140b406-079e-4601-935e-db73bafb4d8e,DISK] 2023-08-27 15:59:45,944 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37545,DS-a11783fe-967d-4589-a49d-d8f189794513,DISK] 2023-08-27 15:59:45,946 INFO [Listener at localhost/40339] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017/jenkins-hbase4.apache.org%2C38379%2C1693151924017.1693151975832 with entries=3, filesize=1.94 KB; new WAL /user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017/jenkins-hbase4.apache.org%2C38379%2C1693151924017.1693151985930 2023-08-27 15:59:45,946 DEBUG [Listener at localhost/40339] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42655,DS-b140b406-079e-4601-935e-db73bafb4d8e,DISK], DatanodeInfoWithStorage[127.0.0.1:37545,DS-a11783fe-967d-4589-a49d-d8f189794513,DISK]] 2023-08-27 15:59:45,946 DEBUG [Listener at localhost/40339] wal.AbstractFSWAL(716): hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017/jenkins-hbase4.apache.org%2C38379%2C1693151924017.1693151975832 is not closed yet, will try archiving it next time 2023-08-27 15:59:45,947 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-27 15:59:45,947 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017/jenkins-hbase4.apache.org%2C38379%2C1693151924017.1693151965691 to hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/oldWALs/jenkins-hbase4.apache.org%2C38379%2C1693151924017.1693151965691 2023-08-27 15:59:45,947 INFO [Listener at localhost/40339] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-27 15:59:45,947 DEBUG [Listener at localhost/40339] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0ed62c52 to 127.0.0.1:62874 2023-08-27 15:59:45,947 DEBUG [Listener at localhost/40339] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 15:59:45,948 DEBUG [Listener at localhost/40339] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-27 15:59:45,948 DEBUG [Listener at localhost/40339] util.JVMClusterUtil(257): Found active master hash=1161861381, stopped=false 2023-08-27 15:59:45,948 INFO [Listener at localhost/40339] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,40183,1693151923973 2023-08-27 15:59:45,950 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-27 15:59:45,950 INFO [Listener at localhost/40339] procedure2.ProcedureExecutor(629): Stopping 2023-08-27 15:59:45,950 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-27 15:59:45,950 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:59:45,950 DEBUG [Listener at localhost/40339] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x73588b3e to 127.0.0.1:62874 2023-08-27 15:59:45,951 DEBUG [Listener at localhost/40339] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 15:59:45,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 15:59:45,951 INFO [Listener at localhost/40339] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38379,1693151924017' ***** 2023-08-27 15:59:45,951 INFO [Listener at localhost/40339] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-27 15:59:45,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 15:59:45,951 INFO [RS:0;jenkins-hbase4:38379] regionserver.HeapMemoryManager(220): Stopping 2023-08-27 15:59:45,951 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-27 15:59:45,951 INFO [RS:0;jenkins-hbase4:38379] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-27 15:59:45,952 INFO [RS:0;jenkins-hbase4:38379] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-27 15:59:45,952 INFO [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(3305): Received CLOSE for 1da8c0df0cf8bfdcb0dcf4ea915d7051 2023-08-27 15:59:45,952 INFO [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(3305): Received CLOSE for 12a0d634325e65d9423a1f5e9f94adb7 2023-08-27 15:59:45,952 INFO [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:45,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1da8c0df0cf8bfdcb0dcf4ea915d7051, disabling compactions & flushes 2023-08-27 15:59:45,952 DEBUG [RS:0;jenkins-hbase4:38379] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x49cf898d to 127.0.0.1:62874 2023-08-27 15:59:45,952 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:59:45,952 DEBUG [RS:0;jenkins-hbase4:38379] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 15:59:45,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:59:45,952 INFO [RS:0;jenkins-hbase4:38379] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-27 15:59:45,953 INFO [RS:0;jenkins-hbase4:38379] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-27 15:59:45,953 INFO [RS:0;jenkins-hbase4:38379] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-27 15:59:45,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. after waiting 0 ms 2023-08-27 15:59:45,953 INFO [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-27 15:59:45,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:59:45,953 INFO [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-08-27 15:59:45,953 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1da8c0df0cf8bfdcb0dcf4ea915d7051 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-27 15:59:45,953 DEBUG [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(1478): Online Regions={1da8c0df0cf8bfdcb0dcf4ea915d7051=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051., 1588230740=hbase:meta,,1.1588230740, 12a0d634325e65d9423a1f5e9f94adb7=hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7.} 2023-08-27 15:59:45,953 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-27 15:59:45,953 DEBUG [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(1504): Waiting on 12a0d634325e65d9423a1f5e9f94adb7, 1588230740, 1da8c0df0cf8bfdcb0dcf4ea915d7051 2023-08-27 15:59:45,953 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-27 15:59:45,953 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-27 15:59:45,953 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-27 15:59:45,954 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-27 15:59:45,954 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.10 KB heapSize=5.61 KB 2023-08-27 15:59:45,977 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=22 (bloomFilter=true), to=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/.tmp/info/2e87550a6bc0437e8651735a2b062537 2023-08-27 15:59:45,979 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.84 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/.tmp/info/1313909fa95a4c1cb7b4ab0b86821d44 2023-08-27 15:59:45,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/.tmp/info/2e87550a6bc0437e8651735a2b062537 as hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/2e87550a6bc0437e8651735a2b062537 2023-08-27 15:59:45,996 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/2e87550a6bc0437e8651735a2b062537, entries=1, sequenceid=22, filesize=5.8 K 2023-08-27 15:59:45,997 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 1da8c0df0cf8bfdcb0dcf4ea915d7051 in 44ms, sequenceid=22, compaction requested=true 2023-08-27 15:59:46,004 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/2fc40b1c6e094b2e9698174ae746c101, hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/59a2d566bf3a4b96a18d278bd7a9d9b8, hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/3812ee15e2894aaeadcb9aaa02ee8b73] to archive 2023-08-27 15:59:46,007 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-27 15:59:46,008 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=264 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/.tmp/table/1121ba4bdd1a4c028346eb747e502536 2023-08-27 15:59:46,011 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/2fc40b1c6e094b2e9698174ae746c101 to hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/2fc40b1c6e094b2e9698174ae746c101 2023-08-27 15:59:46,012 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/59a2d566bf3a4b96a18d278bd7a9d9b8 to hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/59a2d566bf3a4b96a18d278bd7a9d9b8 2023-08-27 15:59:46,014 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/3812ee15e2894aaeadcb9aaa02ee8b73 to hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/info/3812ee15e2894aaeadcb9aaa02ee8b73 2023-08-27 15:59:46,017 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/.tmp/info/1313909fa95a4c1cb7b4ab0b86821d44 as hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/info/1313909fa95a4c1cb7b4ab0b86821d44 2023-08-27 15:59:46,024 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/info/1313909fa95a4c1cb7b4ab0b86821d44, entries=20, sequenceid=14, filesize=7.6 K 2023-08-27 15:59:46,026 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/.tmp/table/1121ba4bdd1a4c028346eb747e502536 as hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/table/1121ba4bdd1a4c028346eb747e502536 2023-08-27 15:59:46,031 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/table/1121ba4bdd1a4c028346eb747e502536, entries=4, sequenceid=14, filesize=4.9 K 2023-08-27 15:59:46,033 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.10 KB/3174, heapSize ~5.33 KB/5456, currentSize=0 B/0 for 1588230740 in 79ms, sequenceid=14, compaction requested=false 2023-08-27 15:59:46,043 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-08-27 15:59:46,044 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-27 15:59:46,044 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-27 15:59:46,044 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-27 15:59:46,045 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-27 15:59:46,046 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/1da8c0df0cf8bfdcb0dcf4ea915d7051/recovered.edits/25.seqid, newMaxSeqId=25, maxSeqId=1 2023-08-27 15:59:46,047 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:59:46,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1da8c0df0cf8bfdcb0dcf4ea915d7051: 2023-08-27 15:59:46,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693151925400.1da8c0df0cf8bfdcb0dcf4ea915d7051. 2023-08-27 15:59:46,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 12a0d634325e65d9423a1f5e9f94adb7, disabling compactions & flushes 2023-08-27 15:59:46,047 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. 2023-08-27 15:59:46,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. 2023-08-27 15:59:46,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. after waiting 0 ms 2023-08-27 15:59:46,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. 2023-08-27 15:59:46,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/data/hbase/namespace/12a0d634325e65d9423a1f5e9f94adb7/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-27 15:59:46,053 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. 2023-08-27 15:59:46,053 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 12a0d634325e65d9423a1f5e9f94adb7: 2023-08-27 15:59:46,053 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693151924728.12a0d634325e65d9423a1f5e9f94adb7. 2023-08-27 15:59:46,153 INFO [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38379,1693151924017; all regions closed. 2023-08-27 15:59:46,157 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/WALs/jenkins-hbase4.apache.org,38379,1693151924017/jenkins-hbase4.apache.org%2C38379%2C1693151924017.meta.1693151924659.meta not finished, retry = 0 2023-08-27 15:59:46,260 DEBUG [RS:0;jenkins-hbase4:38379] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/oldWALs 2023-08-27 15:59:46,261 INFO [RS:0;jenkins-hbase4:38379] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38379%2C1693151924017.meta:.meta(num 1693151924659) 2023-08-27 15:59:46,266 DEBUG [RS:0;jenkins-hbase4:38379] wal.AbstractFSWAL(1028): Moved 2 WAL file(s) to /user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/oldWALs 2023-08-27 15:59:46,266 INFO [RS:0;jenkins-hbase4:38379] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38379%2C1693151924017:(num 1693151985930) 2023-08-27 15:59:46,266 DEBUG [RS:0;jenkins-hbase4:38379] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 15:59:46,266 INFO [RS:0;jenkins-hbase4:38379] regionserver.LeaseManager(133): Closed leases 2023-08-27 15:59:46,266 INFO [RS:0;jenkins-hbase4:38379] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-27 15:59:46,267 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-27 15:59:46,267 INFO [RS:0;jenkins-hbase4:38379] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38379 2023-08-27 15:59:46,270 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-27 15:59:46,270 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38379,1693151924017 2023-08-27 15:59:46,270 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-27 15:59:46,272 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38379,1693151924017] 2023-08-27 15:59:46,272 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38379,1693151924017; numProcessing=1 2023-08-27 15:59:46,278 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,38379,1693151924017 already deleted, retry=false 2023-08-27 15:59:46,278 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38379,1693151924017 expired; onlineServers=0 2023-08-27 15:59:46,278 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40183,1693151923973' ***** 2023-08-27 15:59:46,278 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-27 15:59:46,279 DEBUG [M:0;jenkins-hbase4:40183] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@417f0575, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-27 15:59:46,279 INFO [M:0;jenkins-hbase4:40183] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40183,1693151923973 2023-08-27 15:59:46,279 INFO [M:0;jenkins-hbase4:40183] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40183,1693151923973; all regions closed. 2023-08-27 15:59:46,279 DEBUG [M:0;jenkins-hbase4:40183] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 15:59:46,279 DEBUG [M:0;jenkins-hbase4:40183] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-27 15:59:46,279 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-27 15:59:46,279 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693151924215] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693151924215,5,FailOnTimeoutGroup] 2023-08-27 15:59:46,279 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693151924215] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693151924215,5,FailOnTimeoutGroup] 2023-08-27 15:59:46,279 DEBUG [M:0;jenkins-hbase4:40183] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-27 15:59:46,280 INFO [M:0;jenkins-hbase4:40183] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-27 15:59:46,280 INFO [M:0;jenkins-hbase4:40183] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-27 15:59:46,280 INFO [M:0;jenkins-hbase4:40183] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-27 15:59:46,280 DEBUG [M:0;jenkins-hbase4:40183] master.HMaster(1512): Stopping service threads 2023-08-27 15:59:46,280 INFO [M:0;jenkins-hbase4:40183] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-27 15:59:46,280 ERROR [M:0;jenkins-hbase4:40183] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-27 15:59:46,280 INFO [M:0;jenkins-hbase4:40183] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-27 15:59:46,281 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-27 15:59:46,281 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-27 15:59:46,281 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:59:46,281 DEBUG [M:0;jenkins-hbase4:40183] zookeeper.ZKUtil(398): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-27 15:59:46,281 WARN [M:0;jenkins-hbase4:40183] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-27 15:59:46,281 INFO [M:0;jenkins-hbase4:40183] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-27 15:59:46,281 INFO [M:0;jenkins-hbase4:40183] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-27 15:59:46,281 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-27 15:59:46,282 DEBUG [M:0;jenkins-hbase4:40183] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-27 15:59:46,282 INFO [M:0;jenkins-hbase4:40183] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:59:46,282 DEBUG [M:0;jenkins-hbase4:40183] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:59:46,282 DEBUG [M:0;jenkins-hbase4:40183] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-27 15:59:46,282 DEBUG [M:0;jenkins-hbase4:40183] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:59:46,282 INFO [M:0;jenkins-hbase4:40183] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.88 KB heapSize=47.33 KB 2023-08-27 15:59:46,295 INFO [M:0;jenkins-hbase4:40183] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.88 KB at sequenceid=100 (bloomFilter=true), to=hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d60c8b7e47474d6bb0fddf4d78da859a 2023-08-27 15:59:46,300 INFO [M:0;jenkins-hbase4:40183] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d60c8b7e47474d6bb0fddf4d78da859a 2023-08-27 15:59:46,301 DEBUG [M:0;jenkins-hbase4:40183] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d60c8b7e47474d6bb0fddf4d78da859a as hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d60c8b7e47474d6bb0fddf4d78da859a 2023-08-27 15:59:46,306 INFO [M:0;jenkins-hbase4:40183] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d60c8b7e47474d6bb0fddf4d78da859a 2023-08-27 15:59:46,306 INFO [M:0;jenkins-hbase4:40183] regionserver.HStore(1080): Added hdfs://localhost:36261/user/jenkins/test-data/6a277488-e5e9-59ac-9cf5-a2726b224a18/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d60c8b7e47474d6bb0fddf4d78da859a, entries=11, sequenceid=100, filesize=6.1 K 2023-08-27 15:59:46,308 INFO [M:0;jenkins-hbase4:40183] regionserver.HRegion(2948): Finished flush of dataSize ~38.88 KB/39812, heapSize ~47.31 KB/48448, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 25ms, sequenceid=100, compaction requested=false 2023-08-27 15:59:46,309 INFO [M:0;jenkins-hbase4:40183] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:59:46,309 DEBUG [M:0;jenkins-hbase4:40183] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-27 15:59:46,312 INFO [M:0;jenkins-hbase4:40183] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-27 15:59:46,312 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-27 15:59:46,312 INFO [M:0;jenkins-hbase4:40183] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40183 2023-08-27 15:59:46,314 DEBUG [M:0;jenkins-hbase4:40183] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,40183,1693151923973 already deleted, retry=false 2023-08-27 15:59:46,331 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-27 15:59:46,372 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 15:59:46,372 INFO [RS:0;jenkins-hbase4:38379] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38379,1693151924017; zookeeper connection closed. 2023-08-27 15:59:46,372 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): regionserver:38379-0x1004e7746960001, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 15:59:46,372 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@d957953] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@d957953 2023-08-27 15:59:46,373 INFO [Listener at localhost/40339] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-27 15:59:46,472 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 15:59:46,472 INFO [M:0;jenkins-hbase4:40183] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40183,1693151923973; zookeeper connection closed. 2023-08-27 15:59:46,472 DEBUG [Listener at localhost/40339-EventThread] zookeeper.ZKWatcher(600): master:40183-0x1004e7746960000, quorum=127.0.0.1:62874, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 15:59:46,473 WARN [Listener at localhost/40339] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-27 15:59:46,477 INFO [Listener at localhost/40339] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 15:59:46,581 WARN [BP-1052930590-172.31.14.131-1693151923225 heartbeating to localhost/127.0.0.1:36261] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-27 15:59:46,581 WARN [BP-1052930590-172.31.14.131-1693151923225 heartbeating to localhost/127.0.0.1:36261] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1052930590-172.31.14.131-1693151923225 (Datanode Uuid 56abea07-b38c-4926-a0a9-d545a435eda9) service to localhost/127.0.0.1:36261 2023-08-27 15:59:46,582 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/cluster_7303c16a-7635-dfea-a971-3a5fd5f1b5c3/dfs/data/data3/current/BP-1052930590-172.31.14.131-1693151923225] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:59:46,582 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/cluster_7303c16a-7635-dfea-a971-3a5fd5f1b5c3/dfs/data/data4/current/BP-1052930590-172.31.14.131-1693151923225] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:59:46,583 WARN [Listener at localhost/40339] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-27 15:59:46,586 INFO [Listener at localhost/40339] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 15:59:46,689 WARN [BP-1052930590-172.31.14.131-1693151923225 heartbeating to localhost/127.0.0.1:36261] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-27 15:59:46,689 WARN [BP-1052930590-172.31.14.131-1693151923225 heartbeating to localhost/127.0.0.1:36261] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1052930590-172.31.14.131-1693151923225 (Datanode Uuid 0ca1b14c-0312-4ed1-afda-aeac18008fb4) service to localhost/127.0.0.1:36261 2023-08-27 15:59:46,690 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/cluster_7303c16a-7635-dfea-a971-3a5fd5f1b5c3/dfs/data/data1/current/BP-1052930590-172.31.14.131-1693151923225] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:59:46,690 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/cluster_7303c16a-7635-dfea-a971-3a5fd5f1b5c3/dfs/data/data2/current/BP-1052930590-172.31.14.131-1693151923225] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 15:59:46,702 INFO [Listener at localhost/40339] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 15:59:46,816 INFO [Listener at localhost/40339] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-27 15:59:46,836 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-27 15:59:46,845 INFO [Listener at localhost/40339] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=63 (was 78), OpenFileDescriptor=438 (was 467), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=128 (was 156), ProcessCount=167 (was 169), AvailableMemoryMB=12714 (was 12147) - AvailableMemoryMB LEAK? - 2023-08-27 15:59:46,853 INFO [Listener at localhost/40339] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=64, OpenFileDescriptor=438, MaxFileDescriptor=60000, SystemLoadAverage=128, ProcessCount=167, AvailableMemoryMB=12713 2023-08-27 15:59:46,853 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-27 15:59:46,853 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/hadoop.log.dir so I do NOT create it in target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c 2023-08-27 15:59:46,854 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b6da24b9-b1ce-4125-1091-f0feeefac7c8/hadoop.tmp.dir so I do NOT create it in target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c 2023-08-27 15:59:46,854 INFO [Listener at localhost/40339] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/cluster_270e8e64-5914-4b90-10eb-e150c3ad6967, deleteOnExit=true 2023-08-27 15:59:46,854 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-27 15:59:46,854 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/test.cache.data in system properties and HBase conf 2023-08-27 15:59:46,854 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/hadoop.tmp.dir in system properties and HBase conf 2023-08-27 15:59:46,854 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/hadoop.log.dir in system properties and HBase conf 2023-08-27 15:59:46,854 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-27 15:59:46,854 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-27 15:59:46,854 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-27 15:59:46,855 DEBUG [Listener at localhost/40339] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-27 15:59:46,855 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-27 15:59:46,855 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-27 15:59:46,855 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-27 15:59:46,855 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-27 15:59:46,855 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-27 15:59:46,855 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-27 15:59:46,855 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-27 15:59:46,855 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-27 15:59:46,856 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-27 15:59:46,856 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/nfs.dump.dir in system properties and HBase conf 2023-08-27 15:59:46,856 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/java.io.tmpdir in system properties and HBase conf 2023-08-27 15:59:46,856 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-27 15:59:46,856 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-27 15:59:46,856 INFO [Listener at localhost/40339] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-27 15:59:46,860 WARN [Listener at localhost/40339] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-27 15:59:46,861 WARN [Listener at localhost/40339] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-27 15:59:46,903 WARN [Listener at localhost/40339] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 15:59:46,906 INFO [Listener at localhost/40339] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 15:59:46,911 INFO [Listener at localhost/40339] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/java.io.tmpdir/Jetty_localhost_44777_hdfs____.vrurnj/webapp 2023-08-27 15:59:47,004 INFO [Listener at localhost/40339] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44777 2023-08-27 15:59:47,008 WARN [Listener at localhost/40339] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-27 15:59:47,008 WARN [Listener at localhost/40339] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-27 15:59:47,047 WARN [Listener at localhost/43051] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 15:59:47,061 WARN [Listener at localhost/43051] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-27 15:59:47,063 WARN [Listener at localhost/43051] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 15:59:47,064 INFO [Listener at localhost/43051] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 15:59:47,069 INFO [Listener at localhost/43051] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/java.io.tmpdir/Jetty_localhost_43027_datanode____85h6rh/webapp 2023-08-27 15:59:47,160 INFO [Listener at localhost/43051] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43027 2023-08-27 15:59:47,166 WARN [Listener at localhost/46349] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 15:59:47,179 WARN [Listener at localhost/46349] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-27 15:59:47,181 WARN [Listener at localhost/46349] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 15:59:47,182 INFO [Listener at localhost/46349] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 15:59:47,190 INFO [Listener at localhost/46349] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/java.io.tmpdir/Jetty_localhost_40577_datanode____.pq2eo0/webapp 2023-08-27 15:59:47,265 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfa5e94787fe1840d: Processing first storage report for DS-3fb80ea1-371d-42b2-aa62-98e4f887811f from datanode a6217e0f-3e35-41c2-a839-0de9a7ffe5aa 2023-08-27 15:59:47,265 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfa5e94787fe1840d: from storage DS-3fb80ea1-371d-42b2-aa62-98e4f887811f node DatanodeRegistration(127.0.0.1:33665, datanodeUuid=a6217e0f-3e35-41c2-a839-0de9a7ffe5aa, infoPort=43283, infoSecurePort=0, ipcPort=46349, storageInfo=lv=-57;cid=testClusterID;nsid=763981246;c=1693151986863), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 15:59:47,266 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfa5e94787fe1840d: Processing first storage report for DS-db65e32e-3c3e-420d-84b4-e68aca6c185b from datanode a6217e0f-3e35-41c2-a839-0de9a7ffe5aa 2023-08-27 15:59:47,266 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfa5e94787fe1840d: from storage DS-db65e32e-3c3e-420d-84b4-e68aca6c185b node DatanodeRegistration(127.0.0.1:33665, datanodeUuid=a6217e0f-3e35-41c2-a839-0de9a7ffe5aa, infoPort=43283, infoSecurePort=0, ipcPort=46349, storageInfo=lv=-57;cid=testClusterID;nsid=763981246;c=1693151986863), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 15:59:47,289 INFO [Listener at localhost/46349] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40577 2023-08-27 15:59:47,295 WARN [Listener at localhost/34587] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 15:59:47,379 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa1c64059056de808: Processing first storage report for DS-910a01d4-3a6e-4dbb-b98c-a2a4dd767525 from datanode fd80fdff-7dc0-425b-ae63-fbd550d276e3 2023-08-27 15:59:47,379 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa1c64059056de808: from storage DS-910a01d4-3a6e-4dbb-b98c-a2a4dd767525 node DatanodeRegistration(127.0.0.1:34599, datanodeUuid=fd80fdff-7dc0-425b-ae63-fbd550d276e3, infoPort=44859, infoSecurePort=0, ipcPort=34587, storageInfo=lv=-57;cid=testClusterID;nsid=763981246;c=1693151986863), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 15:59:47,379 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa1c64059056de808: Processing first storage report for DS-eb9dd52c-93e2-4b20-953e-261d53652d31 from datanode fd80fdff-7dc0-425b-ae63-fbd550d276e3 2023-08-27 15:59:47,379 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa1c64059056de808: from storage DS-eb9dd52c-93e2-4b20-953e-261d53652d31 node DatanodeRegistration(127.0.0.1:34599, datanodeUuid=fd80fdff-7dc0-425b-ae63-fbd550d276e3, infoPort=44859, infoSecurePort=0, ipcPort=34587, storageInfo=lv=-57;cid=testClusterID;nsid=763981246;c=1693151986863), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 15:59:47,403 DEBUG [Listener at localhost/34587] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c 2023-08-27 15:59:47,405 INFO [Listener at localhost/34587] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/cluster_270e8e64-5914-4b90-10eb-e150c3ad6967/zookeeper_0, clientPort=61735, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/cluster_270e8e64-5914-4b90-10eb-e150c3ad6967/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/cluster_270e8e64-5914-4b90-10eb-e150c3ad6967/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-27 15:59:47,407 INFO [Listener at localhost/34587] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=61735 2023-08-27 15:59:47,407 INFO [Listener at localhost/34587] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:59:47,408 INFO [Listener at localhost/34587] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:59:47,421 INFO [Listener at localhost/34587] util.FSUtils(471): Created version file at hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b with version=8 2023-08-27 15:59:47,421 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/hbase-staging 2023-08-27 15:59:47,422 INFO [Listener at localhost/34587] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-27 15:59:47,423 INFO [Listener at localhost/34587] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 15:59:47,423 INFO [Listener at localhost/34587] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-27 15:59:47,423 INFO [Listener at localhost/34587] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-27 15:59:47,423 INFO [Listener at localhost/34587] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 15:59:47,423 INFO [Listener at localhost/34587] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-27 15:59:47,423 INFO [Listener at localhost/34587] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-27 15:59:47,424 INFO [Listener at localhost/34587] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43725 2023-08-27 15:59:47,424 INFO [Listener at localhost/34587] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:59:47,425 INFO [Listener at localhost/34587] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:59:47,426 INFO [Listener at localhost/34587] zookeeper.RecoverableZooKeeper(93): Process identifier=master:43725 connecting to ZooKeeper ensemble=127.0.0.1:61735 2023-08-27 15:59:47,433 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:437250x0, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-27 15:59:47,434 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:43725-0x1004e783e730000 connected 2023-08-27 15:59:47,451 DEBUG [Listener at localhost/34587] zookeeper.ZKUtil(164): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-27 15:59:47,452 DEBUG [Listener at localhost/34587] zookeeper.ZKUtil(164): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 15:59:47,452 DEBUG [Listener at localhost/34587] zookeeper.ZKUtil(164): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-27 15:59:47,453 DEBUG [Listener at localhost/34587] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43725 2023-08-27 15:59:47,453 DEBUG [Listener at localhost/34587] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43725 2023-08-27 15:59:47,453 DEBUG [Listener at localhost/34587] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43725 2023-08-27 15:59:47,453 DEBUG [Listener at localhost/34587] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43725 2023-08-27 15:59:47,454 DEBUG [Listener at localhost/34587] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43725 2023-08-27 15:59:47,454 INFO [Listener at localhost/34587] master.HMaster(444): hbase.rootdir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b, hbase.cluster.distributed=false 2023-08-27 15:59:47,466 INFO [Listener at localhost/34587] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-27 15:59:47,466 INFO [Listener at localhost/34587] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 15:59:47,466 INFO [Listener at localhost/34587] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-27 15:59:47,466 INFO [Listener at localhost/34587] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-27 15:59:47,467 INFO [Listener at localhost/34587] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 15:59:47,467 INFO [Listener at localhost/34587] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-27 15:59:47,467 INFO [Listener at localhost/34587] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-27 15:59:47,468 INFO [Listener at localhost/34587] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36373 2023-08-27 15:59:47,468 INFO [Listener at localhost/34587] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-27 15:59:47,469 DEBUG [Listener at localhost/34587] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-27 15:59:47,469 INFO [Listener at localhost/34587] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:59:47,470 INFO [Listener at localhost/34587] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:59:47,471 INFO [Listener at localhost/34587] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36373 connecting to ZooKeeper ensemble=127.0.0.1:61735 2023-08-27 15:59:47,474 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): regionserver:363730x0, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-27 15:59:47,475 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36373-0x1004e783e730001 connected 2023-08-27 15:59:47,475 DEBUG [Listener at localhost/34587] zookeeper.ZKUtil(164): regionserver:36373-0x1004e783e730001, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-27 15:59:47,476 DEBUG [Listener at localhost/34587] zookeeper.ZKUtil(164): regionserver:36373-0x1004e783e730001, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 15:59:47,476 DEBUG [Listener at localhost/34587] zookeeper.ZKUtil(164): regionserver:36373-0x1004e783e730001, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-27 15:59:47,476 DEBUG [Listener at localhost/34587] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36373 2023-08-27 15:59:47,477 DEBUG [Listener at localhost/34587] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36373 2023-08-27 15:59:47,477 DEBUG [Listener at localhost/34587] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36373 2023-08-27 15:59:47,477 DEBUG [Listener at localhost/34587] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36373 2023-08-27 15:59:47,478 DEBUG [Listener at localhost/34587] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36373 2023-08-27 15:59:47,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,43725,1693151987422 2023-08-27 15:59:47,480 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-27 15:59:47,480 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,43725,1693151987422 2023-08-27 15:59:47,481 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-27 15:59:47,481 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): regionserver:36373-0x1004e783e730001, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-27 15:59:47,482 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:59:47,482 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-27 15:59:47,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,43725,1693151987422 from backup master directory 2023-08-27 15:59:47,483 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-27 15:59:47,484 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,43725,1693151987422 2023-08-27 15:59:47,484 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-27 15:59:47,484 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-27 15:59:47,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,43725,1693151987422 2023-08-27 15:59:47,505 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/hbase.id with ID: 332bc997-eba0-4c34-a94e-a632df65cb45 2023-08-27 15:59:47,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:59:47,523 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:59:47,531 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x1bde31fb to 127.0.0.1:61735 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-27 15:59:47,534 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@70133178, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-27 15:59:47,534 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-27 15:59:47,534 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-27 15:59:47,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-27 15:59:47,538 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/MasterData/data/master/store-tmp 2023-08-27 15:59:47,550 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:59:47,550 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-27 15:59:47,550 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:59:47,550 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:59:47,550 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-27 15:59:47,550 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:59:47,551 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 15:59:47,551 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-27 15:59:47,551 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/MasterData/WALs/jenkins-hbase4.apache.org,43725,1693151987422 2023-08-27 15:59:47,554 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43725%2C1693151987422, suffix=, logDir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/MasterData/WALs/jenkins-hbase4.apache.org,43725,1693151987422, archiveDir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/MasterData/oldWALs, maxLogs=10 2023-08-27 15:59:47,572 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33665,DS-3fb80ea1-371d-42b2-aa62-98e4f887811f,DISK] 2023-08-27 15:59:47,573 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34599,DS-910a01d4-3a6e-4dbb-b98c-a2a4dd767525,DISK] 2023-08-27 15:59:47,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/MasterData/WALs/jenkins-hbase4.apache.org,43725,1693151987422/jenkins-hbase4.apache.org%2C43725%2C1693151987422.1693151987554 2023-08-27 15:59:47,575 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33665,DS-3fb80ea1-371d-42b2-aa62-98e4f887811f,DISK], DatanodeInfoWithStorage[127.0.0.1:34599,DS-910a01d4-3a6e-4dbb-b98c-a2a4dd767525,DISK]] 2023-08-27 15:59:47,575 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-27 15:59:47,575 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:59:47,576 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:59:47,576 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:59:47,577 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:59:47,578 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-27 15:59:47,579 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-27 15:59:47,579 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:59:47,580 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:59:47,581 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:59:47,583 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-27 15:59:47,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-27 15:59:47,585 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=819086, jitterRate=0.04152281582355499}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-27 15:59:47,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-27 15:59:47,586 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-27 15:59:47,586 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-27 15:59:47,586 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-27 15:59:47,587 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-27 15:59:47,587 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-27 15:59:47,587 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-08-27 15:59:47,587 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-27 15:59:47,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-27 15:59:47,590 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-27 15:59:47,600 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-27 15:59:47,600 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-27 15:59:47,600 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-27 15:59:47,600 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-27 15:59:47,601 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-27 15:59:47,602 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:59:47,603 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-27 15:59:47,603 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-27 15:59:47,604 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-27 15:59:47,605 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-27 15:59:47,605 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): regionserver:36373-0x1004e783e730001, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-27 15:59:47,605 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:59:47,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,43725,1693151987422, sessionid=0x1004e783e730000, setting cluster-up flag (Was=false) 2023-08-27 15:59:47,610 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:59:47,614 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-27 15:59:47,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43725,1693151987422 2023-08-27 15:59:47,618 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:59:47,622 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-27 15:59:47,622 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43725,1693151987422 2023-08-27 15:59:47,623 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/.hbase-snapshot/.tmp 2023-08-27 15:59:47,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-27 15:59:47,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 15:59:47,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 15:59:47,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 15:59:47,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 15:59:47,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-27 15:59:47,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:59:47,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-27 15:59:47,626 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:59:47,626 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693152017626 2023-08-27 15:59:47,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-27 15:59:47,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-27 15:59:47,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-27 15:59:47,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-27 15:59:47,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-27 15:59:47,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-27 15:59:47,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:47,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-27 15:59:47,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-27 15:59:47,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-27 15:59:47,628 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-27 15:59:47,628 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-27 15:59:47,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-27 15:59:47,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-27 15:59:47,629 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-27 15:59:47,632 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693151987628,5,FailOnTimeoutGroup] 2023-08-27 15:59:47,633 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693151987632,5,FailOnTimeoutGroup] 2023-08-27 15:59:47,633 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:47,633 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-27 15:59:47,633 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:47,633 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:47,641 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-27 15:59:47,642 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-27 15:59:47,642 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b 2023-08-27 15:59:47,650 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:59:47,651 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-27 15:59:47,652 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/info 2023-08-27 15:59:47,653 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-27 15:59:47,653 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:59:47,653 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-27 15:59:47,655 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/rep_barrier 2023-08-27 15:59:47,655 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-27 15:59:47,655 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:59:47,655 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-27 15:59:47,657 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/table 2023-08-27 15:59:47,657 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-27 15:59:47,657 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:59:47,658 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740 2023-08-27 15:59:47,659 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740 2023-08-27 15:59:47,661 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-27 15:59:47,662 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-27 15:59:47,665 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-27 15:59:47,665 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=757726, jitterRate=-0.03650215268135071}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-27 15:59:47,665 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-27 15:59:47,665 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-27 15:59:47,665 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-27 15:59:47,665 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-27 15:59:47,665 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-27 15:59:47,666 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-27 15:59:47,666 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-27 15:59:47,666 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-27 15:59:47,667 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-27 15:59:47,667 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-27 15:59:47,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-27 15:59:47,669 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-27 15:59:47,670 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-27 15:59:47,679 INFO [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(951): ClusterId : 332bc997-eba0-4c34-a94e-a632df65cb45 2023-08-27 15:59:47,680 DEBUG [RS:0;jenkins-hbase4:36373] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-27 15:59:47,681 DEBUG [RS:0;jenkins-hbase4:36373] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-27 15:59:47,681 DEBUG [RS:0;jenkins-hbase4:36373] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-27 15:59:47,683 DEBUG [RS:0;jenkins-hbase4:36373] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-27 15:59:47,684 DEBUG [RS:0;jenkins-hbase4:36373] zookeeper.ReadOnlyZKClient(139): Connect 0x18e738e2 to 127.0.0.1:61735 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-27 15:59:47,689 DEBUG [RS:0;jenkins-hbase4:36373] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1620d58e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-27 15:59:47,689 DEBUG [RS:0;jenkins-hbase4:36373] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@121f5392, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-27 15:59:47,697 DEBUG [RS:0;jenkins-hbase4:36373] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:36373 2023-08-27 15:59:47,697 INFO [RS:0;jenkins-hbase4:36373] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-27 15:59:47,698 INFO [RS:0;jenkins-hbase4:36373] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-27 15:59:47,698 DEBUG [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(1022): About to register with Master. 2023-08-27 15:59:47,698 INFO [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,43725,1693151987422 with isa=jenkins-hbase4.apache.org/172.31.14.131:36373, startcode=1693151987466 2023-08-27 15:59:47,698 DEBUG [RS:0;jenkins-hbase4:36373] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-27 15:59:47,700 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44101, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-08-27 15:59:47,701 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=43725] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 15:59:47,702 DEBUG [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b 2023-08-27 15:59:47,702 DEBUG [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:43051 2023-08-27 15:59:47,702 DEBUG [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-27 15:59:47,703 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-27 15:59:47,704 DEBUG [RS:0;jenkins-hbase4:36373] zookeeper.ZKUtil(162): regionserver:36373-0x1004e783e730001, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 15:59:47,704 WARN [RS:0;jenkins-hbase4:36373] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-27 15:59:47,704 INFO [RS:0;jenkins-hbase4:36373] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-27 15:59:47,704 DEBUG [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(1948): logDir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/WALs/jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 15:59:47,704 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36373,1693151987466] 2023-08-27 15:59:47,707 DEBUG [RS:0;jenkins-hbase4:36373] zookeeper.ZKUtil(162): regionserver:36373-0x1004e783e730001, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 15:59:47,708 DEBUG [RS:0;jenkins-hbase4:36373] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-27 15:59:47,708 INFO [RS:0;jenkins-hbase4:36373] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-27 15:59:47,709 INFO [RS:0;jenkins-hbase4:36373] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-27 15:59:47,710 INFO [RS:0;jenkins-hbase4:36373] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-27 15:59:47,710 INFO [RS:0;jenkins-hbase4:36373] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:47,710 INFO [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-27 15:59:47,711 INFO [RS:0;jenkins-hbase4:36373] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:47,712 DEBUG [RS:0;jenkins-hbase4:36373] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:59:47,712 DEBUG [RS:0;jenkins-hbase4:36373] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:59:47,712 DEBUG [RS:0;jenkins-hbase4:36373] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:59:47,712 DEBUG [RS:0;jenkins-hbase4:36373] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:59:47,712 DEBUG [RS:0;jenkins-hbase4:36373] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:59:47,712 DEBUG [RS:0;jenkins-hbase4:36373] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-27 15:59:47,712 DEBUG [RS:0;jenkins-hbase4:36373] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:59:47,712 DEBUG [RS:0;jenkins-hbase4:36373] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:59:47,712 DEBUG [RS:0;jenkins-hbase4:36373] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:59:47,712 DEBUG [RS:0;jenkins-hbase4:36373] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 15:59:47,713 INFO [RS:0;jenkins-hbase4:36373] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:47,713 INFO [RS:0;jenkins-hbase4:36373] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:47,713 INFO [RS:0;jenkins-hbase4:36373] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:47,724 INFO [RS:0;jenkins-hbase4:36373] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-27 15:59:47,724 INFO [RS:0;jenkins-hbase4:36373] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36373,1693151987466-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:47,734 INFO [RS:0;jenkins-hbase4:36373] regionserver.Replication(203): jenkins-hbase4.apache.org,36373,1693151987466 started 2023-08-27 15:59:47,734 INFO [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,36373,1693151987466, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36373, sessionid=0x1004e783e730001 2023-08-27 15:59:47,734 DEBUG [RS:0;jenkins-hbase4:36373] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-27 15:59:47,734 DEBUG [RS:0;jenkins-hbase4:36373] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 15:59:47,734 DEBUG [RS:0;jenkins-hbase4:36373] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36373,1693151987466' 2023-08-27 15:59:47,734 DEBUG [RS:0;jenkins-hbase4:36373] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-27 15:59:47,735 DEBUG [RS:0;jenkins-hbase4:36373] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-27 15:59:47,735 DEBUG [RS:0;jenkins-hbase4:36373] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-27 15:59:47,735 DEBUG [RS:0;jenkins-hbase4:36373] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-27 15:59:47,735 DEBUG [RS:0;jenkins-hbase4:36373] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 15:59:47,735 DEBUG [RS:0;jenkins-hbase4:36373] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36373,1693151987466' 2023-08-27 15:59:47,735 DEBUG [RS:0;jenkins-hbase4:36373] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-27 15:59:47,735 DEBUG [RS:0;jenkins-hbase4:36373] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-27 15:59:47,736 DEBUG [RS:0;jenkins-hbase4:36373] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-27 15:59:47,736 INFO [RS:0;jenkins-hbase4:36373] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-27 15:59:47,736 INFO [RS:0;jenkins-hbase4:36373] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-27 15:59:47,820 DEBUG [jenkins-hbase4:43725] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-27 15:59:47,821 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36373,1693151987466, state=OPENING 2023-08-27 15:59:47,822 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-27 15:59:47,823 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:59:47,824 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36373,1693151987466}] 2023-08-27 15:59:47,824 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-27 15:59:47,837 INFO [RS:0;jenkins-hbase4:36373] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36373%2C1693151987466, suffix=, logDir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/WALs/jenkins-hbase4.apache.org,36373,1693151987466, archiveDir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/oldWALs, maxLogs=32 2023-08-27 15:59:47,853 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33665,DS-3fb80ea1-371d-42b2-aa62-98e4f887811f,DISK] 2023-08-27 15:59:47,853 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34599,DS-910a01d4-3a6e-4dbb-b98c-a2a4dd767525,DISK] 2023-08-27 15:59:47,855 INFO [RS:0;jenkins-hbase4:36373] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/WALs/jenkins-hbase4.apache.org,36373,1693151987466/jenkins-hbase4.apache.org%2C36373%2C1693151987466.1693151987838 2023-08-27 15:59:47,855 DEBUG [RS:0;jenkins-hbase4:36373] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33665,DS-3fb80ea1-371d-42b2-aa62-98e4f887811f,DISK], DatanodeInfoWithStorage[127.0.0.1:34599,DS-910a01d4-3a6e-4dbb-b98c-a2a4dd767525,DISK]] 2023-08-27 15:59:47,978 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 15:59:47,978 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-27 15:59:47,980 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33420, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-27 15:59:47,984 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-27 15:59:47,984 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-27 15:59:47,986 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36373%2C1693151987466.meta, suffix=.meta, logDir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/WALs/jenkins-hbase4.apache.org,36373,1693151987466, archiveDir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/oldWALs, maxLogs=32 2023-08-27 15:59:48,000 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34599,DS-910a01d4-3a6e-4dbb-b98c-a2a4dd767525,DISK] 2023-08-27 15:59:48,000 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33665,DS-3fb80ea1-371d-42b2-aa62-98e4f887811f,DISK] 2023-08-27 15:59:48,005 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/WALs/jenkins-hbase4.apache.org,36373,1693151987466/jenkins-hbase4.apache.org%2C36373%2C1693151987466.meta.1693151987986.meta 2023-08-27 15:59:48,005 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34599,DS-910a01d4-3a6e-4dbb-b98c-a2a4dd767525,DISK], DatanodeInfoWithStorage[127.0.0.1:33665,DS-3fb80ea1-371d-42b2-aa62-98e4f887811f,DISK]] 2023-08-27 15:59:48,005 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-27 15:59:48,006 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-27 15:59:48,006 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-27 15:59:48,006 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-27 15:59:48,006 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-27 15:59:48,006 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:59:48,006 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-27 15:59:48,006 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-27 15:59:48,008 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-27 15:59:48,009 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/info 2023-08-27 15:59:48,009 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/info 2023-08-27 15:59:48,009 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-27 15:59:48,010 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:59:48,010 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-27 15:59:48,011 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/rep_barrier 2023-08-27 15:59:48,011 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/rep_barrier 2023-08-27 15:59:48,011 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-27 15:59:48,012 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:59:48,012 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-27 15:59:48,013 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/table 2023-08-27 15:59:48,013 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/table 2023-08-27 15:59:48,013 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-27 15:59:48,014 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:59:48,014 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740 2023-08-27 15:59:48,016 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740 2023-08-27 15:59:48,018 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-27 15:59:48,019 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-27 15:59:48,020 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=700568, jitterRate=-0.10918240249156952}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-27 15:59:48,020 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-27 15:59:48,020 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693151987978 2023-08-27 15:59:48,024 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-27 15:59:48,024 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-27 15:59:48,025 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36373,1693151987466, state=OPEN 2023-08-27 15:59:48,027 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-27 15:59:48,027 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-27 15:59:48,029 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-27 15:59:48,029 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36373,1693151987466 in 203 msec 2023-08-27 15:59:48,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-27 15:59:48,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 362 msec 2023-08-27 15:59:48,032 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 407 msec 2023-08-27 15:59:48,032 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693151988032, completionTime=-1 2023-08-27 15:59:48,033 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-27 15:59:48,033 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-27 15:59:48,035 DEBUG [hconnection-0xc365d00-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-27 15:59:48,036 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33422, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-27 15:59:48,038 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-27 15:59:48,038 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693152048038 2023-08-27 15:59:48,038 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693152108038 2023-08-27 15:59:48,038 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-27 15:59:48,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43725,1693151987422-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:48,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43725,1693151987422-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:48,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43725,1693151987422-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:48,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:43725, period=300000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:48,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-27 15:59:48,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-27 15:59:48,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-27 15:59:48,044 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-27 15:59:48,044 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-27 15:59:48,045 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-27 15:59:48,046 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-27 15:59:48,048 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/.tmp/data/hbase/namespace/a7456e6e46ef1322ab6e33eddb1fa562 2023-08-27 15:59:48,048 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/.tmp/data/hbase/namespace/a7456e6e46ef1322ab6e33eddb1fa562 empty. 2023-08-27 15:59:48,049 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/.tmp/data/hbase/namespace/a7456e6e46ef1322ab6e33eddb1fa562 2023-08-27 15:59:48,049 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-27 15:59:48,060 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-27 15:59:48,062 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => a7456e6e46ef1322ab6e33eddb1fa562, NAME => 'hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/.tmp 2023-08-27 15:59:48,070 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:59:48,070 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing a7456e6e46ef1322ab6e33eddb1fa562, disabling compactions & flushes 2023-08-27 15:59:48,070 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562. 2023-08-27 15:59:48,070 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562. 2023-08-27 15:59:48,070 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562. after waiting 0 ms 2023-08-27 15:59:48,070 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562. 2023-08-27 15:59:48,070 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562. 2023-08-27 15:59:48,070 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for a7456e6e46ef1322ab6e33eddb1fa562: 2023-08-27 15:59:48,072 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-27 15:59:48,073 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693151988073"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693151988073"}]},"ts":"1693151988073"} 2023-08-27 15:59:48,075 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-27 15:59:48,076 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-27 15:59:48,077 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693151988076"}]},"ts":"1693151988076"} 2023-08-27 15:59:48,078 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-27 15:59:48,084 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=a7456e6e46ef1322ab6e33eddb1fa562, ASSIGN}] 2023-08-27 15:59:48,086 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=a7456e6e46ef1322ab6e33eddb1fa562, ASSIGN 2023-08-27 15:59:48,086 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=a7456e6e46ef1322ab6e33eddb1fa562, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36373,1693151987466; forceNewPlan=false, retain=false 2023-08-27 15:59:48,237 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=a7456e6e46ef1322ab6e33eddb1fa562, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 15:59:48,238 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693151988237"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693151988237"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693151988237"}]},"ts":"1693151988237"} 2023-08-27 15:59:48,240 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure a7456e6e46ef1322ab6e33eddb1fa562, server=jenkins-hbase4.apache.org,36373,1693151987466}] 2023-08-27 15:59:48,396 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562. 2023-08-27 15:59:48,396 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a7456e6e46ef1322ab6e33eddb1fa562, NAME => 'hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562.', STARTKEY => '', ENDKEY => ''} 2023-08-27 15:59:48,396 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace a7456e6e46ef1322ab6e33eddb1fa562 2023-08-27 15:59:48,396 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:59:48,396 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a7456e6e46ef1322ab6e33eddb1fa562 2023-08-27 15:59:48,396 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a7456e6e46ef1322ab6e33eddb1fa562 2023-08-27 15:59:48,398 INFO [StoreOpener-a7456e6e46ef1322ab6e33eddb1fa562-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region a7456e6e46ef1322ab6e33eddb1fa562 2023-08-27 15:59:48,399 DEBUG [StoreOpener-a7456e6e46ef1322ab6e33eddb1fa562-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/namespace/a7456e6e46ef1322ab6e33eddb1fa562/info 2023-08-27 15:59:48,399 DEBUG [StoreOpener-a7456e6e46ef1322ab6e33eddb1fa562-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/namespace/a7456e6e46ef1322ab6e33eddb1fa562/info 2023-08-27 15:59:48,400 INFO [StoreOpener-a7456e6e46ef1322ab6e33eddb1fa562-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a7456e6e46ef1322ab6e33eddb1fa562 columnFamilyName info 2023-08-27 15:59:48,400 INFO [StoreOpener-a7456e6e46ef1322ab6e33eddb1fa562-1] regionserver.HStore(310): Store=a7456e6e46ef1322ab6e33eddb1fa562/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:59:48,401 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/namespace/a7456e6e46ef1322ab6e33eddb1fa562 2023-08-27 15:59:48,402 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/namespace/a7456e6e46ef1322ab6e33eddb1fa562 2023-08-27 15:59:48,404 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a7456e6e46ef1322ab6e33eddb1fa562 2023-08-27 15:59:48,406 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/namespace/a7456e6e46ef1322ab6e33eddb1fa562/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-27 15:59:48,407 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a7456e6e46ef1322ab6e33eddb1fa562; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=883517, jitterRate=0.12345035374164581}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-27 15:59:48,407 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a7456e6e46ef1322ab6e33eddb1fa562: 2023-08-27 15:59:48,408 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562., pid=6, masterSystemTime=1693151988392 2023-08-27 15:59:48,410 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562. 2023-08-27 15:59:48,410 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562. 2023-08-27 15:59:48,411 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=a7456e6e46ef1322ab6e33eddb1fa562, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 15:59:48,411 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693151988411"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693151988411"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693151988411"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693151988411"}]},"ts":"1693151988411"} 2023-08-27 15:59:48,415 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-27 15:59:48,415 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure a7456e6e46ef1322ab6e33eddb1fa562, server=jenkins-hbase4.apache.org,36373,1693151987466 in 173 msec 2023-08-27 15:59:48,417 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-27 15:59:48,417 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=a7456e6e46ef1322ab6e33eddb1fa562, ASSIGN in 331 msec 2023-08-27 15:59:48,418 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-27 15:59:48,418 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693151988418"}]},"ts":"1693151988418"} 2023-08-27 15:59:48,419 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-27 15:59:48,421 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-27 15:59:48,423 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 378 msec 2023-08-27 15:59:48,445 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-27 15:59:48,446 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-27 15:59:48,446 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:59:48,450 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-27 15:59:48,458 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-27 15:59:48,462 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 11 msec 2023-08-27 15:59:48,472 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-27 15:59:48,482 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-27 15:59:48,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 13 msec 2023-08-27 15:59:48,498 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-27 15:59:48,500 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-27 15:59:48,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.016sec 2023-08-27 15:59:48,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-27 15:59:48,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-27 15:59:48,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-27 15:59:48,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43725,1693151987422-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-27 15:59:48,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43725,1693151987422-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-27 15:59:48,502 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-27 15:59:48,580 DEBUG [Listener at localhost/34587] zookeeper.ReadOnlyZKClient(139): Connect 0x7ef94be7 to 127.0.0.1:61735 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-27 15:59:48,585 DEBUG [Listener at localhost/34587] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@b803e06, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-27 15:59:48,587 DEBUG [hconnection-0x35484e4e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-27 15:59:48,589 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33426, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-27 15:59:48,591 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,43725,1693151987422 2023-08-27 15:59:48,591 INFO [Listener at localhost/34587] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 15:59:48,594 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-27 15:59:48,594 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 15:59:48,595 INFO [Listener at localhost/34587] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-27 15:59:48,596 DEBUG [Listener at localhost/34587] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-27 15:59:48,598 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56284, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-27 15:59:48,599 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43725] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-08-27 15:59:48,600 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43725] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-08-27 15:59:48,600 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43725] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-27 15:59:48,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43725] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRolling 2023-08-27 15:59:48,604 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-08-27 15:59:48,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43725] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRolling" procId is: 9 2023-08-27 15:59:48,605 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-27 15:59:48,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43725] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-27 15:59:48,606 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/.tmp/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd 2023-08-27 15:59:48,607 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/.tmp/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd empty. 2023-08-27 15:59:48,607 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/.tmp/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd 2023-08-27 15:59:48,607 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRolling regions 2023-08-27 15:59:48,618 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/.tmp/data/default/TestLogRolling-testLogRolling/.tabledesc/.tableinfo.0000000001 2023-08-27 15:59:48,619 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => d9d6e24498cd564078afc1ad355276dd, NAME => 'TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/.tmp 2023-08-27 15:59:48,627 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:59:48,627 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1604): Closing d9d6e24498cd564078afc1ad355276dd, disabling compactions & flushes 2023-08-27 15:59:48,627 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. 2023-08-27 15:59:48,627 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. 2023-08-27 15:59:48,627 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. after waiting 0 ms 2023-08-27 15:59:48,627 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. 2023-08-27 15:59:48,627 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. 2023-08-27 15:59:48,627 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1558): Region close journal for d9d6e24498cd564078afc1ad355276dd: 2023-08-27 15:59:48,629 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ADD_TO_META 2023-08-27 15:59:48,631 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693151988631"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693151988631"}]},"ts":"1693151988631"} 2023-08-27 15:59:48,632 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-27 15:59:48,633 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-27 15:59:48,633 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693151988633"}]},"ts":"1693151988633"} 2023-08-27 15:59:48,635 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLING in hbase:meta 2023-08-27 15:59:48,639 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=d9d6e24498cd564078afc1ad355276dd, ASSIGN}] 2023-08-27 15:59:48,641 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=d9d6e24498cd564078afc1ad355276dd, ASSIGN 2023-08-27 15:59:48,642 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=d9d6e24498cd564078afc1ad355276dd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36373,1693151987466; forceNewPlan=false, retain=false 2023-08-27 15:59:48,793 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=d9d6e24498cd564078afc1ad355276dd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 15:59:48,793 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693151988793"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693151988793"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693151988793"}]},"ts":"1693151988793"} 2023-08-27 15:59:48,795 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure d9d6e24498cd564078afc1ad355276dd, server=jenkins-hbase4.apache.org,36373,1693151987466}] 2023-08-27 15:59:48,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. 2023-08-27 15:59:48,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d9d6e24498cd564078afc1ad355276dd, NAME => 'TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.', STARTKEY => '', ENDKEY => ''} 2023-08-27 15:59:48,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling d9d6e24498cd564078afc1ad355276dd 2023-08-27 15:59:48,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 15:59:48,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d9d6e24498cd564078afc1ad355276dd 2023-08-27 15:59:48,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d9d6e24498cd564078afc1ad355276dd 2023-08-27 15:59:48,953 INFO [StoreOpener-d9d6e24498cd564078afc1ad355276dd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region d9d6e24498cd564078afc1ad355276dd 2023-08-27 15:59:48,955 DEBUG [StoreOpener-d9d6e24498cd564078afc1ad355276dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info 2023-08-27 15:59:48,955 DEBUG [StoreOpener-d9d6e24498cd564078afc1ad355276dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info 2023-08-27 15:59:48,955 INFO [StoreOpener-d9d6e24498cd564078afc1ad355276dd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d9d6e24498cd564078afc1ad355276dd columnFamilyName info 2023-08-27 15:59:48,956 INFO [StoreOpener-d9d6e24498cd564078afc1ad355276dd-1] regionserver.HStore(310): Store=d9d6e24498cd564078afc1ad355276dd/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 15:59:48,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd 2023-08-27 15:59:48,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd 2023-08-27 15:59:48,959 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d9d6e24498cd564078afc1ad355276dd 2023-08-27 15:59:48,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-27 15:59:48,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d9d6e24498cd564078afc1ad355276dd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=872448, jitterRate=0.1093754768371582}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-27 15:59:48,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d9d6e24498cd564078afc1ad355276dd: 2023-08-27 15:59:48,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd., pid=11, masterSystemTime=1693151988948 2023-08-27 15:59:48,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. 2023-08-27 15:59:48,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. 2023-08-27 15:59:48,965 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=d9d6e24498cd564078afc1ad355276dd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 15:59:48,965 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693151988965"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693151988965"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693151988965"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693151988965"}]},"ts":"1693151988965"} 2023-08-27 15:59:48,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-27 15:59:48,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure d9d6e24498cd564078afc1ad355276dd, server=jenkins-hbase4.apache.org,36373,1693151987466 in 172 msec 2023-08-27 15:59:48,971 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-27 15:59:48,971 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=d9d6e24498cd564078afc1ad355276dd, ASSIGN in 330 msec 2023-08-27 15:59:48,972 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-27 15:59:48,972 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693151988972"}]},"ts":"1693151988972"} 2023-08-27 15:59:48,973 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLED in hbase:meta 2023-08-27 15:59:48,975 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_POST_OPERATION 2023-08-27 15:59:48,977 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRolling in 375 msec 2023-08-27 15:59:51,570 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-27 15:59:53,708 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-27 15:59:53,709 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-27 15:59:53,709 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRolling' 2023-08-27 15:59:58,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43725] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-27 15:59:58,606 INFO [Listener at localhost/34587] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRolling, procId: 9 completed 2023-08-27 15:59:58,609 DEBUG [Listener at localhost/34587] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRolling 2023-08-27 15:59:58,609 DEBUG [Listener at localhost/34587] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. 2023-08-27 15:59:58,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d9d6e24498cd564078afc1ad355276dd 2023-08-27 15:59:58,623 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d9d6e24498cd564078afc1ad355276dd 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-27 15:59:58,637 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=12 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp/info/82b0855209a04b90b7ecfeecb6b7cbb6 2023-08-27 15:59:58,646 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp/info/82b0855209a04b90b7ecfeecb6b7cbb6 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/82b0855209a04b90b7ecfeecb6b7cbb6 2023-08-27 15:59:58,652 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/82b0855209a04b90b7ecfeecb6b7cbb6, entries=8, sequenceid=12, filesize=13.0 K 2023-08-27 15:59:58,653 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=20.53 KB/21020 for d9d6e24498cd564078afc1ad355276dd in 30ms, sequenceid=12, compaction requested=false 2023-08-27 15:59:58,654 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d9d6e24498cd564078afc1ad355276dd: 2023-08-27 15:59:58,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d9d6e24498cd564078afc1ad355276dd 2023-08-27 15:59:58,654 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d9d6e24498cd564078afc1ad355276dd 1/1 column families, dataSize=21.55 KB heapSize=23.38 KB 2023-08-27 15:59:58,671 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=21.55 KB at sequenceid=36 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp/info/0770f4404eeb4f1db52dd23fc5b60aca 2023-08-27 15:59:58,677 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp/info/0770f4404eeb4f1db52dd23fc5b60aca as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/0770f4404eeb4f1db52dd23fc5b60aca 2023-08-27 15:59:58,683 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/0770f4404eeb4f1db52dd23fc5b60aca, entries=21, sequenceid=36, filesize=26.3 K 2023-08-27 15:59:58,684 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~21.55 KB/22071, heapSize ~23.37 KB/23928, currentSize=3.08 KB/3153 for d9d6e24498cd564078afc1ad355276dd in 30ms, sequenceid=36, compaction requested=false 2023-08-27 15:59:58,684 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d9d6e24498cd564078afc1ad355276dd: 2023-08-27 15:59:58,684 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=39.3 K, sizeToCheck=16.0 K 2023-08-27 15:59:58,684 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-27 15:59:58,685 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/0770f4404eeb4f1db52dd23fc5b60aca because midkey is the same as first or last row 2023-08-27 16:00:00,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d9d6e24498cd564078afc1ad355276dd 2023-08-27 16:00:00,666 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d9d6e24498cd564078afc1ad355276dd 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-27 16:00:00,684 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=47 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp/info/f352e8efc0b945459cbad981c8484735 2023-08-27 16:00:00,691 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp/info/f352e8efc0b945459cbad981c8484735 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/f352e8efc0b945459cbad981c8484735 2023-08-27 16:00:00,698 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/f352e8efc0b945459cbad981c8484735, entries=8, sequenceid=47, filesize=13.0 K 2023-08-27 16:00:00,699 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for d9d6e24498cd564078afc1ad355276dd in 33ms, sequenceid=47, compaction requested=true 2023-08-27 16:00:00,699 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d9d6e24498cd564078afc1ad355276dd: 2023-08-27 16:00:00,699 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=52.3 K, sizeToCheck=16.0 K 2023-08-27 16:00:00,699 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-27 16:00:00,699 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/0770f4404eeb4f1db52dd23fc5b60aca because midkey is the same as first or last row 2023-08-27 16:00:00,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d9d6e24498cd564078afc1ad355276dd 2023-08-27 16:00:00,701 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:00:00,701 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-27 16:00:00,701 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d9d6e24498cd564078afc1ad355276dd 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-27 16:00:00,703 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 53524 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-27 16:00:00,703 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1912): d9d6e24498cd564078afc1ad355276dd/info is initiating minor compaction (all files) 2023-08-27 16:00:00,703 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d9d6e24498cd564078afc1ad355276dd/info in TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. 2023-08-27 16:00:00,703 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/82b0855209a04b90b7ecfeecb6b7cbb6, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/0770f4404eeb4f1db52dd23fc5b60aca, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/f352e8efc0b945459cbad981c8484735] into tmpdir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp, totalSize=52.3 K 2023-08-27 16:00:00,704 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 82b0855209a04b90b7ecfeecb6b7cbb6, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1693151998612 2023-08-27 16:00:00,705 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 0770f4404eeb4f1db52dd23fc5b60aca, keycount=21, bloomtype=ROW, size=26.3 K, encoding=NONE, compression=NONE, seqNum=36, earliestPutTs=1693151998624 2023-08-27 16:00:00,705 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting f352e8efc0b945459cbad981c8484735, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1693151998655 2023-08-27 16:00:00,715 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=23.61 KB at sequenceid=73 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp/info/76e340aff798461490a225eef3bc8bc9 2023-08-27 16:00:00,722 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] throttle.PressureAwareThroughputController(145): d9d6e24498cd564078afc1ad355276dd#info#compaction#19 average throughput is 37.09 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-27 16:00:00,722 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp/info/76e340aff798461490a225eef3bc8bc9 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/76e340aff798461490a225eef3bc8bc9 2023-08-27 16:00:00,781 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/76e340aff798461490a225eef3bc8bc9, entries=23, sequenceid=73, filesize=28.4 K 2023-08-27 16:00:00,782 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~23.61 KB/24173, heapSize ~25.57 KB/26184, currentSize=4.11 KB/4204 for d9d6e24498cd564078afc1ad355276dd in 81ms, sequenceid=73, compaction requested=false 2023-08-27 16:00:00,782 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d9d6e24498cd564078afc1ad355276dd: 2023-08-27 16:00:00,782 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=80.7 K, sizeToCheck=16.0 K 2023-08-27 16:00:00,782 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-27 16:00:00,782 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/76e340aff798461490a225eef3bc8bc9 because midkey is the same as first or last row 2023-08-27 16:00:00,787 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp/info/7d3fa3f35fb54bc8aec5289f67a7fc1c as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/7d3fa3f35fb54bc8aec5289f67a7fc1c 2023-08-27 16:00:00,795 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in d9d6e24498cd564078afc1ad355276dd/info of d9d6e24498cd564078afc1ad355276dd into 7d3fa3f35fb54bc8aec5289f67a7fc1c(size=42.9 K), total size for store is 71.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-27 16:00:00,795 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d9d6e24498cd564078afc1ad355276dd: 2023-08-27 16:00:00,796 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd., storeName=d9d6e24498cd564078afc1ad355276dd/info, priority=13, startTime=1693152000701; duration=0sec 2023-08-27 16:00:00,797 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=71.3 K, sizeToCheck=16.0 K 2023-08-27 16:00:00,797 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-27 16:00:00,797 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.StoreUtils(129): cannot split hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/7d3fa3f35fb54bc8aec5289f67a7fc1c because midkey is the same as first or last row 2023-08-27 16:00:00,797 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:00:02,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d9d6e24498cd564078afc1ad355276dd 2023-08-27 16:00:02,714 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d9d6e24498cd564078afc1ad355276dd 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-27 16:00:02,740 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=d9d6e24498cd564078afc1ad355276dd, server=jenkins-hbase4.apache.org,36373,1693151987466 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-27 16:00:02,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] ipc.CallRunner(144): callId: 98 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:33426 deadline: 1693152012739, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=d9d6e24498cd564078afc1ad355276dd, server=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:00:03,128 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=85 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp/info/9a2efdb700cc4fde9418831038024600 2023-08-27 16:00:03,135 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp/info/9a2efdb700cc4fde9418831038024600 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/9a2efdb700cc4fde9418831038024600 2023-08-27 16:00:03,141 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/9a2efdb700cc4fde9418831038024600, entries=8, sequenceid=85, filesize=13.0 K 2023-08-27 16:00:03,142 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for d9d6e24498cd564078afc1ad355276dd in 427ms, sequenceid=85, compaction requested=true 2023-08-27 16:00:03,142 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d9d6e24498cd564078afc1ad355276dd: 2023-08-27 16:00:03,142 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=84.3 K, sizeToCheck=16.0 K 2023-08-27 16:00:03,142 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-27 16:00:03,142 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/7d3fa3f35fb54bc8aec5289f67a7fc1c because midkey is the same as first or last row 2023-08-27 16:00:03,142 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:00:03,142 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-27 16:00:03,143 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 86295 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-27 16:00:03,143 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1912): d9d6e24498cd564078afc1ad355276dd/info is initiating minor compaction (all files) 2023-08-27 16:00:03,143 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d9d6e24498cd564078afc1ad355276dd/info in TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. 2023-08-27 16:00:03,144 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/7d3fa3f35fb54bc8aec5289f67a7fc1c, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/76e340aff798461490a225eef3bc8bc9, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/9a2efdb700cc4fde9418831038024600] into tmpdir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp, totalSize=84.3 K 2023-08-27 16:00:03,144 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 7d3fa3f35fb54bc8aec5289f67a7fc1c, keycount=37, bloomtype=ROW, size=42.9 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1693151998612 2023-08-27 16:00:03,145 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 76e340aff798461490a225eef3bc8bc9, keycount=23, bloomtype=ROW, size=28.4 K, encoding=NONE, compression=NONE, seqNum=73, earliestPutTs=1693152000667 2023-08-27 16:00:03,145 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 9a2efdb700cc4fde9418831038024600, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1693152000702 2023-08-27 16:00:03,158 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] throttle.PressureAwareThroughputController(145): d9d6e24498cd564078afc1ad355276dd#info#compaction#21 average throughput is 17.04 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-27 16:00:03,175 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp/info/dbbf270a0e0845d79f71ec5ef8a49974 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/dbbf270a0e0845d79f71ec5ef8a49974 2023-08-27 16:00:03,181 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in d9d6e24498cd564078afc1ad355276dd/info of d9d6e24498cd564078afc1ad355276dd into dbbf270a0e0845d79f71ec5ef8a49974(size=75.0 K), total size for store is 75.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-27 16:00:03,181 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d9d6e24498cd564078afc1ad355276dd: 2023-08-27 16:00:03,181 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd., storeName=d9d6e24498cd564078afc1ad355276dd/info, priority=13, startTime=1693152003142; duration=0sec 2023-08-27 16:00:03,181 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=75.0 K, sizeToCheck=16.0 K 2023-08-27 16:00:03,181 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-27 16:00:03,182 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit(227): Splitting TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd., compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:00:03,182 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:00:03,184 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=43725] assignment.AssignmentManager(1140): Split request from jenkins-hbase4.apache.org,36373,1693151987466, parent={ENCODED => d9d6e24498cd564078afc1ad355276dd, NAME => 'TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.', STARTKEY => '', ENDKEY => ''} splitKey=row0064 2023-08-27 16:00:03,193 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=43725] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:00:03,199 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=43725] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=d9d6e24498cd564078afc1ad355276dd, daughterA=0844f99c25e5fb66de044b21c7605b94, daughterB=d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:00:03,200 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=d9d6e24498cd564078afc1ad355276dd, daughterA=0844f99c25e5fb66de044b21c7605b94, daughterB=d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:00:03,200 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=d9d6e24498cd564078afc1ad355276dd, daughterA=0844f99c25e5fb66de044b21c7605b94, daughterB=d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:00:03,200 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=d9d6e24498cd564078afc1ad355276dd, daughterA=0844f99c25e5fb66de044b21c7605b94, daughterB=d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:00:03,208 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=d9d6e24498cd564078afc1ad355276dd, UNASSIGN}] 2023-08-27 16:00:03,209 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=d9d6e24498cd564078afc1ad355276dd, UNASSIGN 2023-08-27 16:00:03,210 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=d9d6e24498cd564078afc1ad355276dd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:00:03,210 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693152003210"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693152003210"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693152003210"}]},"ts":"1693152003210"} 2023-08-27 16:00:03,212 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure d9d6e24498cd564078afc1ad355276dd, server=jenkins-hbase4.apache.org,36373,1693151987466}] 2023-08-27 16:00:03,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close d9d6e24498cd564078afc1ad355276dd 2023-08-27 16:00:03,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d9d6e24498cd564078afc1ad355276dd, disabling compactions & flushes 2023-08-27 16:00:03,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. 2023-08-27 16:00:03,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. 2023-08-27 16:00:03,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. after waiting 0 ms 2023-08-27 16:00:03,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. 2023-08-27 16:00:03,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing d9d6e24498cd564078afc1ad355276dd 1/1 column families, dataSize=21.55 KB heapSize=23.38 KB 2023-08-27 16:00:03,383 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=21.55 KB at sequenceid=110 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp/info/535a195e00fd4339b7b3cde2763f21fb 2023-08-27 16:00:03,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.tmp/info/535a195e00fd4339b7b3cde2763f21fb as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/535a195e00fd4339b7b3cde2763f21fb 2023-08-27 16:00:03,395 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/535a195e00fd4339b7b3cde2763f21fb, entries=21, sequenceid=110, filesize=26.3 K 2023-08-27 16:00:03,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~21.55 KB/22071, heapSize ~23.37 KB/23928, currentSize=0 B/0 for d9d6e24498cd564078afc1ad355276dd in 25ms, sequenceid=110, compaction requested=false 2023-08-27 16:00:03,403 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/82b0855209a04b90b7ecfeecb6b7cbb6, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/0770f4404eeb4f1db52dd23fc5b60aca, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/7d3fa3f35fb54bc8aec5289f67a7fc1c, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/f352e8efc0b945459cbad981c8484735, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/76e340aff798461490a225eef3bc8bc9, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/9a2efdb700cc4fde9418831038024600] to archive 2023-08-27 16:00:03,404 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-27 16:00:03,406 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/82b0855209a04b90b7ecfeecb6b7cbb6 to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/82b0855209a04b90b7ecfeecb6b7cbb6 2023-08-27 16:00:03,407 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/0770f4404eeb4f1db52dd23fc5b60aca to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/0770f4404eeb4f1db52dd23fc5b60aca 2023-08-27 16:00:03,408 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/7d3fa3f35fb54bc8aec5289f67a7fc1c to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/7d3fa3f35fb54bc8aec5289f67a7fc1c 2023-08-27 16:00:03,410 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/f352e8efc0b945459cbad981c8484735 to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/f352e8efc0b945459cbad981c8484735 2023-08-27 16:00:03,411 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/76e340aff798461490a225eef3bc8bc9 to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/76e340aff798461490a225eef3bc8bc9 2023-08-27 16:00:03,412 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/9a2efdb700cc4fde9418831038024600 to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/9a2efdb700cc4fde9418831038024600 2023-08-27 16:00:03,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=1 2023-08-27 16:00:03,419 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. 2023-08-27 16:00:03,420 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d9d6e24498cd564078afc1ad355276dd: 2023-08-27 16:00:03,421 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed d9d6e24498cd564078afc1ad355276dd 2023-08-27 16:00:03,422 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=d9d6e24498cd564078afc1ad355276dd, regionState=CLOSED 2023-08-27 16:00:03,423 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693152003422"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693152003422"}]},"ts":"1693152003422"} 2023-08-27 16:00:03,426 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-08-27 16:00:03,427 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure d9d6e24498cd564078afc1ad355276dd, server=jenkins-hbase4.apache.org,36373,1693151987466 in 212 msec 2023-08-27 16:00:03,428 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-08-27 16:00:03,429 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=d9d6e24498cd564078afc1ad355276dd, UNASSIGN in 219 msec 2023-08-27 16:00:03,441 INFO [PEWorker-4] assignment.SplitTableRegionProcedure(694): pid=12 splitting 2 storefiles, region=d9d6e24498cd564078afc1ad355276dd, threads=2 2023-08-27 16:00:03,443 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/535a195e00fd4339b7b3cde2763f21fb for region: d9d6e24498cd564078afc1ad355276dd 2023-08-27 16:00:03,443 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/dbbf270a0e0845d79f71ec5ef8a49974 for region: d9d6e24498cd564078afc1ad355276dd 2023-08-27 16:00:03,454 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/535a195e00fd4339b7b3cde2763f21fb, top=true 2023-08-27 16:00:03,458 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/.splits/d577d35d8d9449b4d45040b9b63fa497/info/TestLogRolling-testLogRolling=d9d6e24498cd564078afc1ad355276dd-535a195e00fd4339b7b3cde2763f21fb for child: d577d35d8d9449b4d45040b9b63fa497, parent: d9d6e24498cd564078afc1ad355276dd 2023-08-27 16:00:03,458 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/535a195e00fd4339b7b3cde2763f21fb for region: d9d6e24498cd564078afc1ad355276dd 2023-08-27 16:00:03,478 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/dbbf270a0e0845d79f71ec5ef8a49974 for region: d9d6e24498cd564078afc1ad355276dd 2023-08-27 16:00:03,479 DEBUG [PEWorker-4] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region d9d6e24498cd564078afc1ad355276dd Daughter A: 1 storefiles, Daughter B: 2 storefiles. 2023-08-27 16:00:03,516 DEBUG [PEWorker-4] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/0844f99c25e5fb66de044b21c7605b94/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=-1 2023-08-27 16:00:03,518 DEBUG [PEWorker-4] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=-1 2023-08-27 16:00:03,520 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693152003520"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1693152003520"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1693152003520"}]},"ts":"1693152003520"} 2023-08-27 16:00:03,520 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693152003520"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693152003520"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693152003520"}]},"ts":"1693152003520"} 2023-08-27 16:00:03,520 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693152003520"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693152003520"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693152003520"}]},"ts":"1693152003520"} 2023-08-27 16:00:03,561 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36373] regionserver.HRegion(9158): Flush requested on 1588230740 2023-08-27 16:00:03,562 DEBUG [MemStoreFlusher.0] regionserver.FlushAllLargeStoresPolicy(69): Since none of the CFs were above the size, flushing all. 2023-08-27 16:00:03,562 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=4.82 KB heapSize=8.36 KB 2023-08-27 16:00:03,571 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=0844f99c25e5fb66de044b21c7605b94, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=d577d35d8d9449b4d45040b9b63fa497, ASSIGN}] 2023-08-27 16:00:03,573 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=d577d35d8d9449b4d45040b9b63fa497, ASSIGN 2023-08-27 16:00:03,574 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=d577d35d8d9449b4d45040b9b63fa497, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36373,1693151987466; forceNewPlan=false, retain=false 2023-08-27 16:00:03,576 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=0844f99c25e5fb66de044b21c7605b94, ASSIGN 2023-08-27 16:00:03,578 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=0844f99c25e5fb66de044b21c7605b94, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36373,1693151987466; forceNewPlan=false, retain=false 2023-08-27 16:00:03,580 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.61 KB at sequenceid=17 (bloomFilter=false), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/.tmp/info/fc1e7708561d4d4bb00f90a1fa84042a 2023-08-27 16:00:03,598 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=216 B at sequenceid=17 (bloomFilter=false), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/.tmp/table/2ebd465a864942c889a1f3f891cd77c8 2023-08-27 16:00:03,605 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/.tmp/info/fc1e7708561d4d4bb00f90a1fa84042a as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/info/fc1e7708561d4d4bb00f90a1fa84042a 2023-08-27 16:00:03,610 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/info/fc1e7708561d4d4bb00f90a1fa84042a, entries=29, sequenceid=17, filesize=8.6 K 2023-08-27 16:00:03,611 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/.tmp/table/2ebd465a864942c889a1f3f891cd77c8 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/table/2ebd465a864942c889a1f3f891cd77c8 2023-08-27 16:00:03,616 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/table/2ebd465a864942c889a1f3f891cd77c8, entries=4, sequenceid=17, filesize=4.8 K 2023-08-27 16:00:03,617 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~4.82 KB/4934, heapSize ~8.08 KB/8272, currentSize=0 B/0 for 1588230740 in 55ms, sequenceid=17, compaction requested=false 2023-08-27 16:00:03,617 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-08-27 16:00:03,725 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=d577d35d8d9449b4d45040b9b63fa497, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:00:03,725 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=0844f99c25e5fb66de044b21c7605b94, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:00:03,725 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693152003725"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693152003725"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693152003725"}]},"ts":"1693152003725"} 2023-08-27 16:00:03,725 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693152003725"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693152003725"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693152003725"}]},"ts":"1693152003725"} 2023-08-27 16:00:03,727 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 0844f99c25e5fb66de044b21c7605b94, server=jenkins-hbase4.apache.org,36373,1693151987466}] 2023-08-27 16:00:03,728 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure d577d35d8d9449b4d45040b9b63fa497, server=jenkins-hbase4.apache.org,36373,1693151987466}] 2023-08-27 16:00:03,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:00:03,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d577d35d8d9449b4d45040b9b63fa497, NAME => 'TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.', STARTKEY => 'row0064', ENDKEY => ''} 2023-08-27 16:00:03,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:00:03,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 16:00:03,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:00:03,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:00:03,885 INFO [StoreOpener-d577d35d8d9449b4d45040b9b63fa497-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:00:03,886 DEBUG [StoreOpener-d577d35d8d9449b4d45040b9b63fa497-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info 2023-08-27 16:00:03,886 DEBUG [StoreOpener-d577d35d8d9449b4d45040b9b63fa497-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info 2023-08-27 16:00:03,887 INFO [StoreOpener-d577d35d8d9449b4d45040b9b63fa497-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d577d35d8d9449b4d45040b9b63fa497 columnFamilyName info 2023-08-27 16:00:03,897 DEBUG [StoreOpener-d577d35d8d9449b4d45040b9b63fa497-1] regionserver.HStore(539): loaded hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/TestLogRolling-testLogRolling=d9d6e24498cd564078afc1ad355276dd-535a195e00fd4339b7b3cde2763f21fb 2023-08-27 16:00:03,904 DEBUG [StoreOpener-d577d35d8d9449b4d45040b9b63fa497-1] regionserver.HStore(539): loaded hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/dbbf270a0e0845d79f71ec5ef8a49974.d9d6e24498cd564078afc1ad355276dd->hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/dbbf270a0e0845d79f71ec5ef8a49974-top 2023-08-27 16:00:03,905 INFO [StoreOpener-d577d35d8d9449b4d45040b9b63fa497-1] regionserver.HStore(310): Store=d577d35d8d9449b4d45040b9b63fa497/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 16:00:03,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:00:03,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:00:03,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:00:03,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d577d35d8d9449b4d45040b9b63fa497; next sequenceid=114; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=837782, jitterRate=0.06529594957828522}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-27 16:00:03,912 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:00:03,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497., pid=18, masterSystemTime=1693152003879 2023-08-27 16:00:03,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:00:03,914 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-08-27 16:00:03,915 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:00:03,915 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1912): d577d35d8d9449b4d45040b9b63fa497/info is initiating minor compaction (all files) 2023-08-27 16:00:03,915 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d577d35d8d9449b4d45040b9b63fa497/info in TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:00:03,915 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/dbbf270a0e0845d79f71ec5ef8a49974.d9d6e24498cd564078afc1ad355276dd->hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/dbbf270a0e0845d79f71ec5ef8a49974-top, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/TestLogRolling-testLogRolling=d9d6e24498cd564078afc1ad355276dd-535a195e00fd4339b7b3cde2763f21fb] into tmpdir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp, totalSize=101.3 K 2023-08-27 16:00:03,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:00:03,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:00:03,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94. 2023-08-27 16:00:03,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0844f99c25e5fb66de044b21c7605b94, NAME => 'TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94.', STARTKEY => '', ENDKEY => 'row0064'} 2023-08-27 16:00:03,916 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting dbbf270a0e0845d79f71ec5ef8a49974.d9d6e24498cd564078afc1ad355276dd, keycount=34, bloomtype=ROW, size=75.0 K, encoding=NONE, compression=NONE, seqNum=86, earliestPutTs=1693151998612 2023-08-27 16:00:03,917 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=d577d35d8d9449b4d45040b9b63fa497, regionState=OPEN, openSeqNum=114, regionLocation=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:00:03,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling 0844f99c25e5fb66de044b21c7605b94 2023-08-27 16:00:03,917 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693152003916"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693152003916"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693152003916"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693152003916"}]},"ts":"1693152003916"} 2023-08-27 16:00:03,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 16:00:03,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0844f99c25e5fb66de044b21c7605b94 2023-08-27 16:00:03,917 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting TestLogRolling-testLogRolling=d9d6e24498cd564078afc1ad355276dd-535a195e00fd4339b7b3cde2763f21fb, keycount=21, bloomtype=ROW, size=26.3 K, encoding=NONE, compression=NONE, seqNum=110, earliestPutTs=1693152002714 2023-08-27 16:00:03,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0844f99c25e5fb66de044b21c7605b94 2023-08-27 16:00:03,919 INFO [StoreOpener-0844f99c25e5fb66de044b21c7605b94-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 0844f99c25e5fb66de044b21c7605b94 2023-08-27 16:00:03,921 DEBUG [StoreOpener-0844f99c25e5fb66de044b21c7605b94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/0844f99c25e5fb66de044b21c7605b94/info 2023-08-27 16:00:03,921 DEBUG [StoreOpener-0844f99c25e5fb66de044b21c7605b94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/0844f99c25e5fb66de044b21c7605b94/info 2023-08-27 16:00:03,922 INFO [StoreOpener-0844f99c25e5fb66de044b21c7605b94-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0844f99c25e5fb66de044b21c7605b94 columnFamilyName info 2023-08-27 16:00:03,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-08-27 16:00:03,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure d577d35d8d9449b4d45040b9b63fa497, server=jenkins-hbase4.apache.org,36373,1693151987466 in 191 msec 2023-08-27 16:00:03,924 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=d577d35d8d9449b4d45040b9b63fa497, ASSIGN in 352 msec 2023-08-27 16:00:03,928 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] throttle.PressureAwareThroughputController(145): d577d35d8d9449b4d45040b9b63fa497#info#compaction#25 average throughput is 26.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-27 16:00:03,933 DEBUG [StoreOpener-0844f99c25e5fb66de044b21c7605b94-1] regionserver.HStore(539): loaded hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/0844f99c25e5fb66de044b21c7605b94/info/dbbf270a0e0845d79f71ec5ef8a49974.d9d6e24498cd564078afc1ad355276dd->hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/dbbf270a0e0845d79f71ec5ef8a49974-bottom 2023-08-27 16:00:03,934 INFO [StoreOpener-0844f99c25e5fb66de044b21c7605b94-1] regionserver.HStore(310): Store=0844f99c25e5fb66de044b21c7605b94/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 16:00:03,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/0844f99c25e5fb66de044b21c7605b94 2023-08-27 16:00:03,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/0844f99c25e5fb66de044b21c7605b94 2023-08-27 16:00:03,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0844f99c25e5fb66de044b21c7605b94 2023-08-27 16:00:03,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0844f99c25e5fb66de044b21c7605b94; next sequenceid=114; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=811101, jitterRate=0.0313691645860672}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-27 16:00:03,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0844f99c25e5fb66de044b21c7605b94: 2023-08-27 16:00:03,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94., pid=17, masterSystemTime=1693152003879 2023-08-27 16:00:03,964 DEBUG [RS:0;jenkins-hbase4:36373-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-08-27 16:00:03,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:00:03,965 INFO [RS:0;jenkins-hbase4:36373-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94. 2023-08-27 16:00:03,965 DEBUG [RS:0;jenkins-hbase4:36373-longCompactions-0] regionserver.HStore(1912): 0844f99c25e5fb66de044b21c7605b94/info is initiating minor compaction (all files) 2023-08-27 16:00:03,965 INFO [RS:0;jenkins-hbase4:36373-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 0844f99c25e5fb66de044b21c7605b94/info in TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94. 2023-08-27 16:00:03,965 INFO [RS:0;jenkins-hbase4:36373-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/0844f99c25e5fb66de044b21c7605b94/info/dbbf270a0e0845d79f71ec5ef8a49974.d9d6e24498cd564078afc1ad355276dd->hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/dbbf270a0e0845d79f71ec5ef8a49974-bottom] into tmpdir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/0844f99c25e5fb66de044b21c7605b94/.tmp, totalSize=75.0 K 2023-08-27 16:00:03,966 DEBUG [RS:0;jenkins-hbase4:36373-longCompactions-0] compactions.Compactor(207): Compacting dbbf270a0e0845d79f71ec5ef8a49974.d9d6e24498cd564078afc1ad355276dd, keycount=34, bloomtype=ROW, size=75.0 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1693151998612 2023-08-27 16:00:03,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94. 2023-08-27 16:00:03,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94. 2023-08-27 16:00:03,968 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=0844f99c25e5fb66de044b21c7605b94, regionState=OPEN, openSeqNum=114, regionLocation=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:00:03,969 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693152003968"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693152003968"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693152003968"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693152003968"}]},"ts":"1693152003968"} 2023-08-27 16:00:03,970 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/ed52bdc948144300ac468e6485938550 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/ed52bdc948144300ac468e6485938550 2023-08-27 16:00:03,973 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-08-27 16:00:03,974 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 0844f99c25e5fb66de044b21c7605b94, server=jenkins-hbase4.apache.org,36373,1693151987466 in 244 msec 2023-08-27 16:00:03,976 INFO [RS:0;jenkins-hbase4:36373-longCompactions-0] throttle.PressureAwareThroughputController(145): 0844f99c25e5fb66de044b21c7605b94#info#compaction#26 average throughput is 21.05 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-27 16:00:03,977 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-08-27 16:00:03,977 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=0844f99c25e5fb66de044b21c7605b94, ASSIGN in 403 msec 2023-08-27 16:00:03,979 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=d9d6e24498cd564078afc1ad355276dd, daughterA=0844f99c25e5fb66de044b21c7605b94, daughterB=d577d35d8d9449b4d45040b9b63fa497 in 785 msec 2023-08-27 16:00:03,983 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 (all) file(s) in d577d35d8d9449b4d45040b9b63fa497/info of d577d35d8d9449b4d45040b9b63fa497 into ed52bdc948144300ac468e6485938550(size=31.6 K), total size for store is 31.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-27 16:00:03,983 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:00:03,983 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497., storeName=d577d35d8d9449b4d45040b9b63fa497/info, priority=14, startTime=1693152003913; duration=0sec 2023-08-27 16:00:03,983 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:00:03,998 DEBUG [RS:0;jenkins-hbase4:36373-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/0844f99c25e5fb66de044b21c7605b94/.tmp/info/14a3a4e45da34054aaacfaf3ec7c2223 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/0844f99c25e5fb66de044b21c7605b94/info/14a3a4e45da34054aaacfaf3ec7c2223 2023-08-27 16:00:04,005 INFO [RS:0;jenkins-hbase4:36373-longCompactions-0] regionserver.HStore(1652): Completed compaction of 1 (all) file(s) in 0844f99c25e5fb66de044b21c7605b94/info of 0844f99c25e5fb66de044b21c7605b94 into 14a3a4e45da34054aaacfaf3ec7c2223(size=69.7 K), total size for store is 69.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-27 16:00:04,005 DEBUG [RS:0;jenkins-hbase4:36373-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0844f99c25e5fb66de044b21c7605b94: 2023-08-27 16:00:04,005 INFO [RS:0;jenkins-hbase4:36373-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94., storeName=0844f99c25e5fb66de044b21c7605b94/info, priority=15, startTime=1693152003964; duration=0sec 2023-08-27 16:00:04,005 DEBUG [RS:0;jenkins-hbase4:36373-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:00:08,953 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-27 16:00:12,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] ipc.CallRunner(144): callId: 100 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:33426 deadline: 1693152022780, exception=org.apache.hadoop.hbase.NotServingRegionException: TestLogRolling-testLogRolling,,1693151988599.d9d6e24498cd564078afc1ad355276dd. is not online on jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:00:34,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:00:34,931 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d577d35d8d9449b4d45040b9b63fa497 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-27 16:00:34,952 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=125 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/49e5647c1cba4530b9bf513d920963ac 2023-08-27 16:00:34,960 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/49e5647c1cba4530b9bf513d920963ac as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/49e5647c1cba4530b9bf513d920963ac 2023-08-27 16:00:34,962 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=d577d35d8d9449b4d45040b9b63fa497, server=jenkins-hbase4.apache.org,36373,1693151987466 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-27 16:00:34,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] ipc.CallRunner(144): callId: 133 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:33426 deadline: 1693152044962, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=d577d35d8d9449b4d45040b9b63fa497, server=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:00:34,965 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/49e5647c1cba4530b9bf513d920963ac, entries=8, sequenceid=125, filesize=13.0 K 2023-08-27 16:00:34,966 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for d577d35d8d9449b4d45040b9b63fa497 in 35ms, sequenceid=125, compaction requested=false 2023-08-27 16:00:34,966 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:00:45,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:00:45,038 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d577d35d8d9449b4d45040b9b63fa497 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-27 16:00:45,050 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=d577d35d8d9449b4d45040b9b63fa497, server=jenkins-hbase4.apache.org,36373,1693151987466 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-27 16:00:45,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] ipc.CallRunner(144): callId: 143 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:33426 deadline: 1693152055050, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=d577d35d8d9449b4d45040b9b63fa497, server=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:00:45,053 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=150 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/695e6b80a3b0465a8999e45f637ed5de 2023-08-27 16:00:45,060 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/695e6b80a3b0465a8999e45f637ed5de as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/695e6b80a3b0465a8999e45f637ed5de 2023-08-27 16:00:45,067 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/695e6b80a3b0465a8999e45f637ed5de, entries=22, sequenceid=150, filesize=27.4 K 2023-08-27 16:00:45,067 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=7.18 KB/7357 for d577d35d8d9449b4d45040b9b63fa497 in 29ms, sequenceid=150, compaction requested=true 2023-08-27 16:00:45,068 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:00:45,068 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:00:45,068 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-27 16:00:45,069 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 73719 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-27 16:00:45,069 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1912): d577d35d8d9449b4d45040b9b63fa497/info is initiating minor compaction (all files) 2023-08-27 16:00:45,069 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d577d35d8d9449b4d45040b9b63fa497/info in TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:00:45,069 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/ed52bdc948144300ac468e6485938550, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/49e5647c1cba4530b9bf513d920963ac, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/695e6b80a3b0465a8999e45f637ed5de] into tmpdir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp, totalSize=72.0 K 2023-08-27 16:00:45,070 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting ed52bdc948144300ac468e6485938550, keycount=26, bloomtype=ROW, size=31.6 K, encoding=NONE, compression=NONE, seqNum=110, earliestPutTs=1693152000706 2023-08-27 16:00:45,070 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 49e5647c1cba4530b9bf513d920963ac, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=125, earliestPutTs=1693152032923 2023-08-27 16:00:45,070 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 695e6b80a3b0465a8999e45f637ed5de, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=150, earliestPutTs=1693152034932 2023-08-27 16:00:45,082 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] throttle.PressureAwareThroughputController(145): d577d35d8d9449b4d45040b9b63fa497#info#compaction#29 average throughput is 28.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-27 16:00:45,099 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/bf2bedd8fe7347a8bdb3654935a56d8b as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/bf2bedd8fe7347a8bdb3654935a56d8b 2023-08-27 16:00:45,105 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in d577d35d8d9449b4d45040b9b63fa497/info of d577d35d8d9449b4d45040b9b63fa497 into bf2bedd8fe7347a8bdb3654935a56d8b(size=62.7 K), total size for store is 62.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-27 16:00:45,105 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:00:45,105 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497., storeName=d577d35d8d9449b4d45040b9b63fa497/info, priority=13, startTime=1693152045068; duration=0sec 2023-08-27 16:00:45,105 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:00:55,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:00:55,059 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d577d35d8d9449b4d45040b9b63fa497 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-27 16:00:55,070 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=162 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/9ab0d4b427ab49b08f660ca142ab8e55 2023-08-27 16:00:55,076 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/9ab0d4b427ab49b08f660ca142ab8e55 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/9ab0d4b427ab49b08f660ca142ab8e55 2023-08-27 16:00:55,080 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/9ab0d4b427ab49b08f660ca142ab8e55, entries=8, sequenceid=162, filesize=13.0 K 2023-08-27 16:00:55,081 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=1.03 KB/1051 for d577d35d8d9449b4d45040b9b63fa497 in 22ms, sequenceid=162, compaction requested=false 2023-08-27 16:00:55,081 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:00:57,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:00:57,069 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d577d35d8d9449b4d45040b9b63fa497 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-27 16:00:57,084 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=173 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/71ec8b43403448ff913587e28bcd0361 2023-08-27 16:00:57,091 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/71ec8b43403448ff913587e28bcd0361 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/71ec8b43403448ff913587e28bcd0361 2023-08-27 16:00:57,096 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=d577d35d8d9449b4d45040b9b63fa497, server=jenkins-hbase4.apache.org,36373,1693151987466 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-27 16:00:57,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] ipc.CallRunner(144): callId: 175 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:33426 deadline: 1693152067096, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=d577d35d8d9449b4d45040b9b63fa497, server=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:00:57,097 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/71ec8b43403448ff913587e28bcd0361, entries=8, sequenceid=173, filesize=13.0 K 2023-08-27 16:00:57,097 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for d577d35d8d9449b4d45040b9b63fa497 in 28ms, sequenceid=173, compaction requested=true 2023-08-27 16:00:57,098 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:00:57,098 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:00:57,098 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-27 16:00:57,099 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 90753 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-27 16:00:57,099 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1912): d577d35d8d9449b4d45040b9b63fa497/info is initiating minor compaction (all files) 2023-08-27 16:00:57,099 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d577d35d8d9449b4d45040b9b63fa497/info in TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:00:57,099 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/bf2bedd8fe7347a8bdb3654935a56d8b, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/9ab0d4b427ab49b08f660ca142ab8e55, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/71ec8b43403448ff913587e28bcd0361] into tmpdir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp, totalSize=88.6 K 2023-08-27 16:00:57,100 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting bf2bedd8fe7347a8bdb3654935a56d8b, keycount=56, bloomtype=ROW, size=62.7 K, encoding=NONE, compression=NONE, seqNum=150, earliestPutTs=1693152000706 2023-08-27 16:00:57,100 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 9ab0d4b427ab49b08f660ca142ab8e55, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=162, earliestPutTs=1693152045039 2023-08-27 16:00:57,100 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 71ec8b43403448ff913587e28bcd0361, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=173, earliestPutTs=1693152055060 2023-08-27 16:00:57,114 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] throttle.PressureAwareThroughputController(145): d577d35d8d9449b4d45040b9b63fa497#info#compaction#32 average throughput is 24.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-27 16:00:57,129 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/2e960bde6a994fea8bc6b5c752b68cdc as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/2e960bde6a994fea8bc6b5c752b68cdc 2023-08-27 16:00:57,135 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in d577d35d8d9449b4d45040b9b63fa497/info of d577d35d8d9449b4d45040b9b63fa497 into 2e960bde6a994fea8bc6b5c752b68cdc(size=79.3 K), total size for store is 79.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-27 16:00:57,135 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:00:57,136 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497., storeName=d577d35d8d9449b4d45040b9b63fa497/info, priority=13, startTime=1693152057098; duration=0sec 2023-08-27 16:00:57,136 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:01:07,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:01:07,149 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d577d35d8d9449b4d45040b9b63fa497 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-27 16:01:07,163 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=199 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/7dedff4ddc16479086833152a4525ea7 2023-08-27 16:01:07,169 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/7dedff4ddc16479086833152a4525ea7 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/7dedff4ddc16479086833152a4525ea7 2023-08-27 16:01:07,174 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/7dedff4ddc16479086833152a4525ea7, entries=22, sequenceid=199, filesize=27.4 K 2023-08-27 16:01:07,174 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=3.08 KB/3153 for d577d35d8d9449b4d45040b9b63fa497 in 25ms, sequenceid=199, compaction requested=false 2023-08-27 16:01:07,175 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:01:09,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:01:09,160 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d577d35d8d9449b4d45040b9b63fa497 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-27 16:01:09,173 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=210 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/63bed6ac9481484c935584a7fdd51bfb 2023-08-27 16:01:09,179 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/63bed6ac9481484c935584a7fdd51bfb as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/63bed6ac9481484c935584a7fdd51bfb 2023-08-27 16:01:09,185 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/63bed6ac9481484c935584a7fdd51bfb, entries=8, sequenceid=210, filesize=13.0 K 2023-08-27 16:01:09,185 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=d577d35d8d9449b4d45040b9b63fa497, server=jenkins-hbase4.apache.org,36373,1693151987466 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-27 16:01:09,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] ipc.CallRunner(144): callId: 207 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:33426 deadline: 1693152079185, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=d577d35d8d9449b4d45040b9b63fa497, server=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:01:09,186 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for d577d35d8d9449b4d45040b9b63fa497 in 26ms, sequenceid=210, compaction requested=true 2023-08-27 16:01:09,186 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:01:09,186 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:01:09,186 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-27 16:01:09,187 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 122538 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-27 16:01:09,187 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1912): d577d35d8d9449b4d45040b9b63fa497/info is initiating minor compaction (all files) 2023-08-27 16:01:09,187 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d577d35d8d9449b4d45040b9b63fa497/info in TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:01:09,187 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/2e960bde6a994fea8bc6b5c752b68cdc, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/7dedff4ddc16479086833152a4525ea7, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/63bed6ac9481484c935584a7fdd51bfb] into tmpdir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp, totalSize=119.7 K 2023-08-27 16:01:09,188 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 2e960bde6a994fea8bc6b5c752b68cdc, keycount=72, bloomtype=ROW, size=79.3 K, encoding=NONE, compression=NONE, seqNum=173, earliestPutTs=1693152000706 2023-08-27 16:01:09,188 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 7dedff4ddc16479086833152a4525ea7, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=199, earliestPutTs=1693152057070 2023-08-27 16:01:09,189 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 63bed6ac9481484c935584a7fdd51bfb, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=210, earliestPutTs=1693152067150 2023-08-27 16:01:09,200 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] throttle.PressureAwareThroughputController(145): d577d35d8d9449b4d45040b9b63fa497#info#compaction#35 average throughput is 51.12 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-27 16:01:09,219 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/5594cd5be9034149b171649d9e362ffc as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/5594cd5be9034149b171649d9e362ffc 2023-08-27 16:01:09,224 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in d577d35d8d9449b4d45040b9b63fa497/info of d577d35d8d9449b4d45040b9b63fa497 into 5594cd5be9034149b171649d9e362ffc(size=110.3 K), total size for store is 110.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-27 16:01:09,224 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:01:09,224 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497., storeName=d577d35d8d9449b4d45040b9b63fa497/info, priority=13, startTime=1693152069186; duration=0sec 2023-08-27 16:01:09,224 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:01:19,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:01:19,249 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d577d35d8d9449b4d45040b9b63fa497 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-27 16:01:19,260 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=236 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/4ceb134b494c4950b72c64ff722709fa 2023-08-27 16:01:19,266 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/4ceb134b494c4950b72c64ff722709fa as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/4ceb134b494c4950b72c64ff722709fa 2023-08-27 16:01:19,271 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/4ceb134b494c4950b72c64ff722709fa, entries=22, sequenceid=236, filesize=27.4 K 2023-08-27 16:01:19,272 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=5.13 KB/5255 for d577d35d8d9449b4d45040b9b63fa497 in 22ms, sequenceid=236, compaction requested=false 2023-08-27 16:01:19,272 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:01:21,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:01:21,261 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d577d35d8d9449b4d45040b9b63fa497 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-27 16:01:21,278 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=247 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/47df7075fb434493a223399a135bceef 2023-08-27 16:01:21,284 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=d577d35d8d9449b4d45040b9b63fa497, server=jenkins-hbase4.apache.org,36373,1693151987466 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-27 16:01:21,284 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/47df7075fb434493a223399a135bceef as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/47df7075fb434493a223399a135bceef 2023-08-27 16:01:21,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:33426 deadline: 1693152091284, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=d577d35d8d9449b4d45040b9b63fa497, server=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:01:21,289 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/47df7075fb434493a223399a135bceef, entries=8, sequenceid=247, filesize=13.0 K 2023-08-27 16:01:21,290 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for d577d35d8d9449b4d45040b9b63fa497 in 29ms, sequenceid=247, compaction requested=true 2023-08-27 16:01:21,290 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:01:21,290 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:01:21,290 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-27 16:01:21,291 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 154238 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-27 16:01:21,291 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1912): d577d35d8d9449b4d45040b9b63fa497/info is initiating minor compaction (all files) 2023-08-27 16:01:21,292 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d577d35d8d9449b4d45040b9b63fa497/info in TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:01:21,292 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/5594cd5be9034149b171649d9e362ffc, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/4ceb134b494c4950b72c64ff722709fa, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/47df7075fb434493a223399a135bceef] into tmpdir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp, totalSize=150.6 K 2023-08-27 16:01:21,292 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 5594cd5be9034149b171649d9e362ffc, keycount=102, bloomtype=ROW, size=110.3 K, encoding=NONE, compression=NONE, seqNum=210, earliestPutTs=1693152000706 2023-08-27 16:01:21,292 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 4ceb134b494c4950b72c64ff722709fa, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=236, earliestPutTs=1693152069161 2023-08-27 16:01:21,293 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 47df7075fb434493a223399a135bceef, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=247, earliestPutTs=1693152079250 2023-08-27 16:01:21,304 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] throttle.PressureAwareThroughputController(145): d577d35d8d9449b4d45040b9b63fa497#info#compaction#38 average throughput is 66.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-27 16:01:21,320 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/e84dffd145bd46759f82b598850a6960 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/e84dffd145bd46759f82b598850a6960 2023-08-27 16:01:21,325 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in d577d35d8d9449b4d45040b9b63fa497/info of d577d35d8d9449b4d45040b9b63fa497 into e84dffd145bd46759f82b598850a6960(size=141.4 K), total size for store is 141.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-27 16:01:21,325 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:01:21,325 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497., storeName=d577d35d8d9449b4d45040b9b63fa497/info, priority=13, startTime=1693152081290; duration=0sec 2023-08-27 16:01:21,325 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:01:31,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:01:31,317 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d577d35d8d9449b4d45040b9b63fa497 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-27 16:01:31,327 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=273 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/b62a9110224d496ea56b2f9f9f318298 2023-08-27 16:01:31,332 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/b62a9110224d496ea56b2f9f9f318298 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/b62a9110224d496ea56b2f9f9f318298 2023-08-27 16:01:31,337 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/b62a9110224d496ea56b2f9f9f318298, entries=22, sequenceid=273, filesize=27.4 K 2023-08-27 16:01:31,338 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=7.18 KB/7357 for d577d35d8d9449b4d45040b9b63fa497 in 21ms, sequenceid=273, compaction requested=false 2023-08-27 16:01:31,338 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:01:33,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:01:33,327 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d577d35d8d9449b4d45040b9b63fa497 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-27 16:01:33,338 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=284 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/74eb502c615744d9bec8d7269dd56f04 2023-08-27 16:01:33,344 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/74eb502c615744d9bec8d7269dd56f04 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/74eb502c615744d9bec8d7269dd56f04 2023-08-27 16:01:33,349 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/74eb502c615744d9bec8d7269dd56f04, entries=8, sequenceid=284, filesize=13.0 K 2023-08-27 16:01:33,350 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for d577d35d8d9449b4d45040b9b63fa497 in 23ms, sequenceid=284, compaction requested=true 2023-08-27 16:01:33,350 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:01:33,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:01:33,350 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-27 16:01:33,350 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:01:33,351 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d577d35d8d9449b4d45040b9b63fa497 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-27 16:01:33,352 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 186157 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-27 16:01:33,352 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1912): d577d35d8d9449b4d45040b9b63fa497/info is initiating minor compaction (all files) 2023-08-27 16:01:33,352 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d577d35d8d9449b4d45040b9b63fa497/info in TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:01:33,352 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/e84dffd145bd46759f82b598850a6960, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/b62a9110224d496ea56b2f9f9f318298, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/74eb502c615744d9bec8d7269dd56f04] into tmpdir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp, totalSize=181.8 K 2023-08-27 16:01:33,353 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting e84dffd145bd46759f82b598850a6960, keycount=132, bloomtype=ROW, size=141.4 K, encoding=NONE, compression=NONE, seqNum=247, earliestPutTs=1693152000706 2023-08-27 16:01:33,353 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting b62a9110224d496ea56b2f9f9f318298, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=273, earliestPutTs=1693152081261 2023-08-27 16:01:33,354 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting 74eb502c615744d9bec8d7269dd56f04, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=284, earliestPutTs=1693152091317 2023-08-27 16:01:33,359 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=d577d35d8d9449b4d45040b9b63fa497, server=jenkins-hbase4.apache.org,36373,1693151987466 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-27 16:01:33,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] ipc.CallRunner(144): callId: 279 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:33426 deadline: 1693152103358, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=d577d35d8d9449b4d45040b9b63fa497, server=jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:01:33,365 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=309 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/a4e21aac038c4b62926582e96027d076 2023-08-27 16:01:33,373 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] throttle.PressureAwareThroughputController(145): d577d35d8d9449b4d45040b9b63fa497#info#compaction#42 average throughput is 32.47 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-27 16:01:33,375 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/a4e21aac038c4b62926582e96027d076 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/a4e21aac038c4b62926582e96027d076 2023-08-27 16:01:33,380 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/a4e21aac038c4b62926582e96027d076, entries=22, sequenceid=309, filesize=27.4 K 2023-08-27 16:01:33,381 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=7.18 KB/7357 for d577d35d8d9449b4d45040b9b63fa497 in 30ms, sequenceid=309, compaction requested=false 2023-08-27 16:01:33,381 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:01:33,399 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/f6865410df7e44d8b54346913d380893 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/f6865410df7e44d8b54346913d380893 2023-08-27 16:01:33,405 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in d577d35d8d9449b4d45040b9b63fa497/info of d577d35d8d9449b4d45040b9b63fa497 into f6865410df7e44d8b54346913d380893(size=172.4 K), total size for store is 199.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-27 16:01:33,405 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:01:33,405 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497., storeName=d577d35d8d9449b4d45040b9b63fa497/info, priority=13, startTime=1693152093350; duration=0sec 2023-08-27 16:01:33,405 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:01:43,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36373] regionserver.HRegion(9158): Flush requested on d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:01:43,429 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing d577d35d8d9449b4d45040b9b63fa497 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-27 16:01:43,444 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=321 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/a4d55ee34ef44d84ab83dfe8f1a6fa3d 2023-08-27 16:01:43,451 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/a4d55ee34ef44d84ab83dfe8f1a6fa3d as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/a4d55ee34ef44d84ab83dfe8f1a6fa3d 2023-08-27 16:01:43,455 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/a4d55ee34ef44d84ab83dfe8f1a6fa3d, entries=8, sequenceid=321, filesize=13.0 K 2023-08-27 16:01:43,456 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=1.03 KB/1051 for d577d35d8d9449b4d45040b9b63fa497 in 27ms, sequenceid=321, compaction requested=true 2023-08-27 16:01:43,456 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:01:43,456 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:01:43,456 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-27 16:01:43,458 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 217887 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-27 16:01:43,458 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1912): d577d35d8d9449b4d45040b9b63fa497/info is initiating minor compaction (all files) 2023-08-27 16:01:43,458 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d577d35d8d9449b4d45040b9b63fa497/info in TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:01:43,458 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/f6865410df7e44d8b54346913d380893, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/a4e21aac038c4b62926582e96027d076, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/a4d55ee34ef44d84ab83dfe8f1a6fa3d] into tmpdir=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp, totalSize=212.8 K 2023-08-27 16:01:43,458 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting f6865410df7e44d8b54346913d380893, keycount=162, bloomtype=ROW, size=172.4 K, encoding=NONE, compression=NONE, seqNum=284, earliestPutTs=1693152000706 2023-08-27 16:01:43,459 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting a4e21aac038c4b62926582e96027d076, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=309, earliestPutTs=1693152093328 2023-08-27 16:01:43,459 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] compactions.Compactor(207): Compacting a4d55ee34ef44d84ab83dfe8f1a6fa3d, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=321, earliestPutTs=1693152093351 2023-08-27 16:01:43,474 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] throttle.PressureAwareThroughputController(145): d577d35d8d9449b4d45040b9b63fa497#info#compaction#44 average throughput is 64.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-27 16:01:43,484 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/995e513bed13452386c5b0f949c60022 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/995e513bed13452386c5b0f949c60022 2023-08-27 16:01:43,489 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in d577d35d8d9449b4d45040b9b63fa497/info of d577d35d8d9449b4d45040b9b63fa497 into 995e513bed13452386c5b0f949c60022(size=203.4 K), total size for store is 203.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-27 16:01:43,489 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:01:43,489 INFO [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497., storeName=d577d35d8d9449b4d45040b9b63fa497/info, priority=13, startTime=1693152103456; duration=0sec 2023-08-27 16:01:43,490 DEBUG [RS:0;jenkins-hbase4:36373-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-27 16:01:45,431 INFO [Listener at localhost/34587] wal.AbstractTestLogRolling(188): after writing there are 0 log files 2023-08-27 16:01:45,449 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34599,DS-910a01d4-3a6e-4dbb-b98c-a2a4dd767525,DISK] 2023-08-27 16:01:45,449 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33665,DS-3fb80ea1-371d-42b2-aa62-98e4f887811f,DISK] 2023-08-27 16:01:45,452 INFO [Listener at localhost/34587] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/WALs/jenkins-hbase4.apache.org,36373,1693151987466/jenkins-hbase4.apache.org%2C36373%2C1693151987466.1693151987838 with entries=309, filesize=300.61 KB; new WAL /user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/WALs/jenkins-hbase4.apache.org,36373,1693151987466/jenkins-hbase4.apache.org%2C36373%2C1693151987466.1693152105432 2023-08-27 16:01:45,453 DEBUG [Listener at localhost/34587] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34599,DS-910a01d4-3a6e-4dbb-b98c-a2a4dd767525,DISK], DatanodeInfoWithStorage[127.0.0.1:33665,DS-3fb80ea1-371d-42b2-aa62-98e4f887811f,DISK]] 2023-08-27 16:01:45,453 DEBUG [Listener at localhost/34587] wal.AbstractFSWAL(716): hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/WALs/jenkins-hbase4.apache.org,36373,1693151987466/jenkins-hbase4.apache.org%2C36373%2C1693151987466.1693151987838 is not closed yet, will try archiving it next time 2023-08-27 16:01:45,458 DEBUG [Listener at localhost/34587] regionserver.HRegion(2446): Flush status journal for 0844f99c25e5fb66de044b21c7605b94: 2023-08-27 16:01:45,458 INFO [Listener at localhost/34587] regionserver.HRegion(2745): Flushing a7456e6e46ef1322ab6e33eddb1fa562 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-27 16:01:45,469 INFO [Listener at localhost/34587] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/namespace/a7456e6e46ef1322ab6e33eddb1fa562/.tmp/info/6434a8827c4a4d84b450eae02939f33d 2023-08-27 16:01:45,474 DEBUG [Listener at localhost/34587] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/namespace/a7456e6e46ef1322ab6e33eddb1fa562/.tmp/info/6434a8827c4a4d84b450eae02939f33d as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/namespace/a7456e6e46ef1322ab6e33eddb1fa562/info/6434a8827c4a4d84b450eae02939f33d 2023-08-27 16:01:45,479 INFO [Listener at localhost/34587] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/namespace/a7456e6e46ef1322ab6e33eddb1fa562/info/6434a8827c4a4d84b450eae02939f33d, entries=2, sequenceid=6, filesize=4.8 K 2023-08-27 16:01:45,480 INFO [Listener at localhost/34587] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for a7456e6e46ef1322ab6e33eddb1fa562 in 22ms, sequenceid=6, compaction requested=false 2023-08-27 16:01:45,481 DEBUG [Listener at localhost/34587] regionserver.HRegion(2446): Flush status journal for a7456e6e46ef1322ab6e33eddb1fa562: 2023-08-27 16:01:45,481 INFO [Listener at localhost/34587] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.26 KB heapSize=4.19 KB 2023-08-27 16:01:45,492 INFO [Listener at localhost/34587] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.26 KB at sequenceid=24 (bloomFilter=false), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/.tmp/info/e0b1f9eec2594edda36b3cd34de4b41f 2023-08-27 16:01:45,497 DEBUG [Listener at localhost/34587] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/.tmp/info/e0b1f9eec2594edda36b3cd34de4b41f as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/info/e0b1f9eec2594edda36b3cd34de4b41f 2023-08-27 16:01:45,502 INFO [Listener at localhost/34587] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/info/e0b1f9eec2594edda36b3cd34de4b41f, entries=16, sequenceid=24, filesize=7.0 K 2023-08-27 16:01:45,503 INFO [Listener at localhost/34587] regionserver.HRegion(2948): Finished flush of dataSize ~2.26 KB/2312, heapSize ~3.67 KB/3760, currentSize=0 B/0 for 1588230740 in 22ms, sequenceid=24, compaction requested=false 2023-08-27 16:01:45,503 DEBUG [Listener at localhost/34587] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-08-27 16:01:45,503 INFO [Listener at localhost/34587] regionserver.HRegion(2745): Flushing d577d35d8d9449b4d45040b9b63fa497 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-27 16:01:45,516 INFO [Listener at localhost/34587] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=326 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/ef40bc36aec44bda8e5eec2a8f8d7598 2023-08-27 16:01:45,521 DEBUG [Listener at localhost/34587] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/.tmp/info/ef40bc36aec44bda8e5eec2a8f8d7598 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/ef40bc36aec44bda8e5eec2a8f8d7598 2023-08-27 16:01:45,525 INFO [Listener at localhost/34587] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/ef40bc36aec44bda8e5eec2a8f8d7598, entries=1, sequenceid=326, filesize=5.8 K 2023-08-27 16:01:45,526 INFO [Listener at localhost/34587] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for d577d35d8d9449b4d45040b9b63fa497 in 23ms, sequenceid=326, compaction requested=false 2023-08-27 16:01:45,526 DEBUG [Listener at localhost/34587] regionserver.HRegion(2446): Flush status journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:01:45,539 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33665,DS-3fb80ea1-371d-42b2-aa62-98e4f887811f,DISK] 2023-08-27 16:01:45,540 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34599,DS-910a01d4-3a6e-4dbb-b98c-a2a4dd767525,DISK] 2023-08-27 16:01:45,542 INFO [Listener at localhost/34587] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/WALs/jenkins-hbase4.apache.org,36373,1693151987466/jenkins-hbase4.apache.org%2C36373%2C1693151987466.1693152105432 with entries=4, filesize=1.22 KB; new WAL /user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/WALs/jenkins-hbase4.apache.org,36373,1693151987466/jenkins-hbase4.apache.org%2C36373%2C1693151987466.1693152105526 2023-08-27 16:01:45,542 DEBUG [Listener at localhost/34587] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33665,DS-3fb80ea1-371d-42b2-aa62-98e4f887811f,DISK], DatanodeInfoWithStorage[127.0.0.1:34599,DS-910a01d4-3a6e-4dbb-b98c-a2a4dd767525,DISK]] 2023-08-27 16:01:45,542 DEBUG [Listener at localhost/34587] wal.AbstractFSWAL(716): hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/WALs/jenkins-hbase4.apache.org,36373,1693151987466/jenkins-hbase4.apache.org%2C36373%2C1693151987466.1693152105432 is not closed yet, will try archiving it next time 2023-08-27 16:01:45,542 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/WALs/jenkins-hbase4.apache.org,36373,1693151987466/jenkins-hbase4.apache.org%2C36373%2C1693151987466.1693151987838 to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/oldWALs/jenkins-hbase4.apache.org%2C36373%2C1693151987466.1693151987838 2023-08-27 16:01:45,543 INFO [Listener at localhost/34587] hbase.Waiter(180): Waiting up to [5,000] milli-secs(wait.for.ratio=[1]) 2023-08-27 16:01:45,545 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/WALs/jenkins-hbase4.apache.org,36373,1693151987466/jenkins-hbase4.apache.org%2C36373%2C1693151987466.1693152105432 to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/oldWALs/jenkins-hbase4.apache.org%2C36373%2C1693151987466.1693152105432 2023-08-27 16:01:45,643 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-27 16:01:45,644 INFO [Listener at localhost/34587] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-27 16:01:45,644 DEBUG [Listener at localhost/34587] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7ef94be7 to 127.0.0.1:61735 2023-08-27 16:01:45,644 DEBUG [Listener at localhost/34587] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 16:01:45,644 DEBUG [Listener at localhost/34587] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-27 16:01:45,644 DEBUG [Listener at localhost/34587] util.JVMClusterUtil(257): Found active master hash=220481562, stopped=false 2023-08-27 16:01:45,644 INFO [Listener at localhost/34587] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,43725,1693151987422 2023-08-27 16:01:45,646 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-27 16:01:45,646 INFO [Listener at localhost/34587] procedure2.ProcedureExecutor(629): Stopping 2023-08-27 16:01:45,646 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 16:01:45,646 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): regionserver:36373-0x1004e783e730001, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-27 16:01:45,646 DEBUG [Listener at localhost/34587] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1bde31fb to 127.0.0.1:61735 2023-08-27 16:01:45,647 DEBUG [Listener at localhost/34587] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 16:01:45,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 16:01:45,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36373-0x1004e783e730001, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 16:01:45,647 INFO [Listener at localhost/34587] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36373,1693151987466' ***** 2023-08-27 16:01:45,647 INFO [Listener at localhost/34587] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-27 16:01:45,648 INFO [RS:0;jenkins-hbase4:36373] regionserver.HeapMemoryManager(220): Stopping 2023-08-27 16:01:45,648 INFO [RS:0;jenkins-hbase4:36373] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-27 16:01:45,648 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-27 16:01:45,648 INFO [RS:0;jenkins-hbase4:36373] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-27 16:01:45,648 INFO [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(3305): Received CLOSE for 0844f99c25e5fb66de044b21c7605b94 2023-08-27 16:01:45,648 INFO [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(3305): Received CLOSE for a7456e6e46ef1322ab6e33eddb1fa562 2023-08-27 16:01:45,648 INFO [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(3305): Received CLOSE for d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:01:45,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0844f99c25e5fb66de044b21c7605b94, disabling compactions & flushes 2023-08-27 16:01:45,648 INFO [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:01:45,648 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94. 2023-08-27 16:01:45,648 DEBUG [RS:0;jenkins-hbase4:36373] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x18e738e2 to 127.0.0.1:61735 2023-08-27 16:01:45,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94. 2023-08-27 16:01:45,648 DEBUG [RS:0;jenkins-hbase4:36373] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 16:01:45,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94. after waiting 0 ms 2023-08-27 16:01:45,648 INFO [RS:0;jenkins-hbase4:36373] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-27 16:01:45,648 INFO [RS:0;jenkins-hbase4:36373] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-27 16:01:45,648 INFO [RS:0;jenkins-hbase4:36373] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-27 16:01:45,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94. 2023-08-27 16:01:45,649 INFO [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-27 16:01:45,649 INFO [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-08-27 16:01:45,649 DEBUG [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(1478): Online Regions={0844f99c25e5fb66de044b21c7605b94=TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94., a7456e6e46ef1322ab6e33eddb1fa562=hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562., 1588230740=hbase:meta,,1.1588230740, d577d35d8d9449b4d45040b9b63fa497=TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.} 2023-08-27 16:01:45,650 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-27 16:01:45,650 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-27 16:01:45,650 DEBUG [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(1504): Waiting on 0844f99c25e5fb66de044b21c7605b94, 1588230740, a7456e6e46ef1322ab6e33eddb1fa562, d577d35d8d9449b4d45040b9b63fa497 2023-08-27 16:01:45,650 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-27 16:01:45,650 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/0844f99c25e5fb66de044b21c7605b94/info/dbbf270a0e0845d79f71ec5ef8a49974.d9d6e24498cd564078afc1ad355276dd->hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/dbbf270a0e0845d79f71ec5ef8a49974-bottom] to archive 2023-08-27 16:01:45,650 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-27 16:01:45,650 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-27 16:01:45,652 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-27 16:01:45,655 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/0844f99c25e5fb66de044b21c7605b94/info/dbbf270a0e0845d79f71ec5ef8a49974.d9d6e24498cd564078afc1ad355276dd to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/0844f99c25e5fb66de044b21c7605b94/info/dbbf270a0e0845d79f71ec5ef8a49974.d9d6e24498cd564078afc1ad355276dd 2023-08-27 16:01:45,658 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/meta/1588230740/recovered.edits/27.seqid, newMaxSeqId=27, maxSeqId=1 2023-08-27 16:01:45,658 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-27 16:01:45,659 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-27 16:01:45,659 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-27 16:01:45,659 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-27 16:01:45,661 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/0844f99c25e5fb66de044b21c7605b94/recovered.edits/118.seqid, newMaxSeqId=118, maxSeqId=113 2023-08-27 16:01:45,662 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94. 2023-08-27 16:01:45,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0844f99c25e5fb66de044b21c7605b94: 2023-08-27 16:01:45,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,,1693152003193.0844f99c25e5fb66de044b21c7605b94. 2023-08-27 16:01:45,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a7456e6e46ef1322ab6e33eddb1fa562, disabling compactions & flushes 2023-08-27 16:01:45,662 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562. 2023-08-27 16:01:45,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562. 2023-08-27 16:01:45,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562. after waiting 0 ms 2023-08-27 16:01:45,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562. 2023-08-27 16:01:45,666 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/hbase/namespace/a7456e6e46ef1322ab6e33eddb1fa562/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-27 16:01:45,667 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562. 2023-08-27 16:01:45,667 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a7456e6e46ef1322ab6e33eddb1fa562: 2023-08-27 16:01:45,667 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693151988043.a7456e6e46ef1322ab6e33eddb1fa562. 2023-08-27 16:01:45,667 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d577d35d8d9449b4d45040b9b63fa497, disabling compactions & flushes 2023-08-27 16:01:45,667 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:01:45,667 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:01:45,667 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. after waiting 0 ms 2023-08-27 16:01:45,667 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:01:45,680 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/dbbf270a0e0845d79f71ec5ef8a49974.d9d6e24498cd564078afc1ad355276dd->hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d9d6e24498cd564078afc1ad355276dd/info/dbbf270a0e0845d79f71ec5ef8a49974-top, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/ed52bdc948144300ac468e6485938550, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/TestLogRolling-testLogRolling=d9d6e24498cd564078afc1ad355276dd-535a195e00fd4339b7b3cde2763f21fb, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/49e5647c1cba4530b9bf513d920963ac, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/bf2bedd8fe7347a8bdb3654935a56d8b, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/695e6b80a3b0465a8999e45f637ed5de, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/9ab0d4b427ab49b08f660ca142ab8e55, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/2e960bde6a994fea8bc6b5c752b68cdc, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/71ec8b43403448ff913587e28bcd0361, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/7dedff4ddc16479086833152a4525ea7, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/5594cd5be9034149b171649d9e362ffc, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/63bed6ac9481484c935584a7fdd51bfb, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/4ceb134b494c4950b72c64ff722709fa, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/e84dffd145bd46759f82b598850a6960, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/47df7075fb434493a223399a135bceef, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/b62a9110224d496ea56b2f9f9f318298, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/f6865410df7e44d8b54346913d380893, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/74eb502c615744d9bec8d7269dd56f04, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/a4e21aac038c4b62926582e96027d076, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/a4d55ee34ef44d84ab83dfe8f1a6fa3d] to archive 2023-08-27 16:01:45,681 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-27 16:01:45,682 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/dbbf270a0e0845d79f71ec5ef8a49974.d9d6e24498cd564078afc1ad355276dd to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/dbbf270a0e0845d79f71ec5ef8a49974.d9d6e24498cd564078afc1ad355276dd 2023-08-27 16:01:45,684 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/ed52bdc948144300ac468e6485938550 to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/ed52bdc948144300ac468e6485938550 2023-08-27 16:01:45,685 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/TestLogRolling-testLogRolling=d9d6e24498cd564078afc1ad355276dd-535a195e00fd4339b7b3cde2763f21fb to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/TestLogRolling-testLogRolling=d9d6e24498cd564078afc1ad355276dd-535a195e00fd4339b7b3cde2763f21fb 2023-08-27 16:01:45,686 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/49e5647c1cba4530b9bf513d920963ac to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/49e5647c1cba4530b9bf513d920963ac 2023-08-27 16:01:45,687 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/bf2bedd8fe7347a8bdb3654935a56d8b to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/bf2bedd8fe7347a8bdb3654935a56d8b 2023-08-27 16:01:45,688 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/695e6b80a3b0465a8999e45f637ed5de to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/695e6b80a3b0465a8999e45f637ed5de 2023-08-27 16:01:45,690 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/9ab0d4b427ab49b08f660ca142ab8e55 to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/9ab0d4b427ab49b08f660ca142ab8e55 2023-08-27 16:01:45,691 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/2e960bde6a994fea8bc6b5c752b68cdc to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/2e960bde6a994fea8bc6b5c752b68cdc 2023-08-27 16:01:45,692 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/71ec8b43403448ff913587e28bcd0361 to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/71ec8b43403448ff913587e28bcd0361 2023-08-27 16:01:45,693 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/7dedff4ddc16479086833152a4525ea7 to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/7dedff4ddc16479086833152a4525ea7 2023-08-27 16:01:45,694 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/5594cd5be9034149b171649d9e362ffc to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/5594cd5be9034149b171649d9e362ffc 2023-08-27 16:01:45,695 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/63bed6ac9481484c935584a7fdd51bfb to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/63bed6ac9481484c935584a7fdd51bfb 2023-08-27 16:01:45,696 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/4ceb134b494c4950b72c64ff722709fa to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/4ceb134b494c4950b72c64ff722709fa 2023-08-27 16:01:45,698 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/e84dffd145bd46759f82b598850a6960 to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/e84dffd145bd46759f82b598850a6960 2023-08-27 16:01:45,699 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/47df7075fb434493a223399a135bceef to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/47df7075fb434493a223399a135bceef 2023-08-27 16:01:45,700 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/b62a9110224d496ea56b2f9f9f318298 to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/b62a9110224d496ea56b2f9f9f318298 2023-08-27 16:01:45,701 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/f6865410df7e44d8b54346913d380893 to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/f6865410df7e44d8b54346913d380893 2023-08-27 16:01:45,702 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/74eb502c615744d9bec8d7269dd56f04 to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/74eb502c615744d9bec8d7269dd56f04 2023-08-27 16:01:45,703 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/a4e21aac038c4b62926582e96027d076 to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/a4e21aac038c4b62926582e96027d076 2023-08-27 16:01:45,704 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/a4d55ee34ef44d84ab83dfe8f1a6fa3d to hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/archive/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/info/a4d55ee34ef44d84ab83dfe8f1a6fa3d 2023-08-27 16:01:45,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/data/default/TestLogRolling-testLogRolling/d577d35d8d9449b4d45040b9b63fa497/recovered.edits/329.seqid, newMaxSeqId=329, maxSeqId=113 2023-08-27 16:01:45,709 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:01:45,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d577d35d8d9449b4d45040b9b63fa497: 2023-08-27 16:01:45,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,row0064,1693152003193.d577d35d8d9449b4d45040b9b63fa497. 2023-08-27 16:01:45,714 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-27 16:01:45,714 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-27 16:01:45,718 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-27 16:01:45,850 INFO [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36373,1693151987466; all regions closed. 2023-08-27 16:01:45,855 DEBUG [RS:0;jenkins-hbase4:36373] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/oldWALs 2023-08-27 16:01:45,855 INFO [RS:0;jenkins-hbase4:36373] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36373%2C1693151987466.meta:.meta(num 1693151987986) 2023-08-27 16:01:45,860 DEBUG [RS:0;jenkins-hbase4:36373] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/oldWALs 2023-08-27 16:01:45,860 INFO [RS:0;jenkins-hbase4:36373] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36373%2C1693151987466:(num 1693152105526) 2023-08-27 16:01:45,860 DEBUG [RS:0;jenkins-hbase4:36373] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 16:01:45,860 INFO [RS:0;jenkins-hbase4:36373] regionserver.LeaseManager(133): Closed leases 2023-08-27 16:01:45,861 INFO [RS:0;jenkins-hbase4:36373] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-27 16:01:45,861 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-27 16:01:45,861 INFO [RS:0;jenkins-hbase4:36373] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36373 2023-08-27 16:01:45,865 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): regionserver:36373-0x1004e783e730001, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36373,1693151987466 2023-08-27 16:01:45,865 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-27 16:01:45,865 ERROR [Listener at localhost/34587-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@54e0e892 rejected from java.util.concurrent.ThreadPoolExecutor@50df76d6[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1374) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:603) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-08-27 16:01:45,865 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): regionserver:36373-0x1004e783e730001, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-27 16:01:45,866 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36373,1693151987466] 2023-08-27 16:01:45,866 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36373,1693151987466; numProcessing=1 2023-08-27 16:01:45,869 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36373,1693151987466 already deleted, retry=false 2023-08-27 16:01:45,869 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36373,1693151987466 expired; onlineServers=0 2023-08-27 16:01:45,869 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43725,1693151987422' ***** 2023-08-27 16:01:45,869 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-27 16:01:45,870 DEBUG [M:0;jenkins-hbase4:43725] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@91bdd66, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-27 16:01:45,870 INFO [M:0;jenkins-hbase4:43725] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43725,1693151987422 2023-08-27 16:01:45,870 INFO [M:0;jenkins-hbase4:43725] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43725,1693151987422; all regions closed. 2023-08-27 16:01:45,870 DEBUG [M:0;jenkins-hbase4:43725] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 16:01:45,870 DEBUG [M:0;jenkins-hbase4:43725] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-27 16:01:45,870 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-27 16:01:45,870 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693151987628] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693151987628,5,FailOnTimeoutGroup] 2023-08-27 16:01:45,870 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693151987632] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693151987632,5,FailOnTimeoutGroup] 2023-08-27 16:01:45,870 DEBUG [M:0;jenkins-hbase4:43725] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-27 16:01:45,871 INFO [M:0;jenkins-hbase4:43725] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-27 16:01:45,871 INFO [M:0;jenkins-hbase4:43725] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-27 16:01:45,871 INFO [M:0;jenkins-hbase4:43725] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-27 16:01:45,871 DEBUG [M:0;jenkins-hbase4:43725] master.HMaster(1512): Stopping service threads 2023-08-27 16:01:45,871 INFO [M:0;jenkins-hbase4:43725] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-27 16:01:45,871 ERROR [M:0;jenkins-hbase4:43725] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-27 16:01:45,872 INFO [M:0;jenkins-hbase4:43725] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-27 16:01:45,872 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-27 16:01:45,872 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-27 16:01:45,872 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 16:01:45,872 DEBUG [M:0;jenkins-hbase4:43725] zookeeper.ZKUtil(398): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-27 16:01:45,872 WARN [M:0;jenkins-hbase4:43725] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-27 16:01:45,872 INFO [M:0;jenkins-hbase4:43725] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-27 16:01:45,872 INFO [M:0;jenkins-hbase4:43725] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-27 16:01:45,873 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-27 16:01:45,873 DEBUG [M:0;jenkins-hbase4:43725] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-27 16:01:45,873 INFO [M:0;jenkins-hbase4:43725] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 16:01:45,873 DEBUG [M:0;jenkins-hbase4:43725] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 16:01:45,873 DEBUG [M:0;jenkins-hbase4:43725] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-27 16:01:45,873 DEBUG [M:0;jenkins-hbase4:43725] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 16:01:45,873 INFO [M:0;jenkins-hbase4:43725] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=64.69 KB heapSize=78.42 KB 2023-08-27 16:01:45,885 INFO [M:0;jenkins-hbase4:43725] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=64.69 KB at sequenceid=160 (bloomFilter=true), to=hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/7d32c35d2a0d41b493424bd7ade85df9 2023-08-27 16:01:45,890 INFO [M:0;jenkins-hbase4:43725] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7d32c35d2a0d41b493424bd7ade85df9 2023-08-27 16:01:45,891 DEBUG [M:0;jenkins-hbase4:43725] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/7d32c35d2a0d41b493424bd7ade85df9 as hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7d32c35d2a0d41b493424bd7ade85df9 2023-08-27 16:01:45,896 INFO [M:0;jenkins-hbase4:43725] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7d32c35d2a0d41b493424bd7ade85df9 2023-08-27 16:01:45,896 INFO [M:0;jenkins-hbase4:43725] regionserver.HStore(1080): Added hdfs://localhost:43051/user/jenkins/test-data/a699bd2d-dc37-7e75-73dc-8acb0ca2a90b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7d32c35d2a0d41b493424bd7ade85df9, entries=18, sequenceid=160, filesize=6.9 K 2023-08-27 16:01:45,897 INFO [M:0;jenkins-hbase4:43725] regionserver.HRegion(2948): Finished flush of dataSize ~64.69 KB/66244, heapSize ~78.41 KB/80288, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 24ms, sequenceid=160, compaction requested=false 2023-08-27 16:01:45,898 INFO [M:0;jenkins-hbase4:43725] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 16:01:45,898 DEBUG [M:0;jenkins-hbase4:43725] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-27 16:01:45,901 INFO [M:0;jenkins-hbase4:43725] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-27 16:01:45,901 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-27 16:01:45,901 INFO [M:0;jenkins-hbase4:43725] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43725 2023-08-27 16:01:45,904 DEBUG [M:0;jenkins-hbase4:43725] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,43725,1693151987422 already deleted, retry=false 2023-08-27 16:01:45,967 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): regionserver:36373-0x1004e783e730001, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 16:01:45,967 INFO [RS:0;jenkins-hbase4:36373] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36373,1693151987466; zookeeper connection closed. 2023-08-27 16:01:45,967 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): regionserver:36373-0x1004e783e730001, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 16:01:45,967 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@dac44f] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@dac44f 2023-08-27 16:01:45,967 INFO [Listener at localhost/34587] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-27 16:01:46,067 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 16:01:46,067 INFO [M:0;jenkins-hbase4:43725] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43725,1693151987422; zookeeper connection closed. 2023-08-27 16:01:46,067 DEBUG [Listener at localhost/34587-EventThread] zookeeper.ZKWatcher(600): master:43725-0x1004e783e730000, quorum=127.0.0.1:61735, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 16:01:46,068 WARN [Listener at localhost/34587] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-27 16:01:46,072 INFO [Listener at localhost/34587] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 16:01:46,177 WARN [BP-931663746-172.31.14.131-1693151986863 heartbeating to localhost/127.0.0.1:43051] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-27 16:01:46,177 WARN [BP-931663746-172.31.14.131-1693151986863 heartbeating to localhost/127.0.0.1:43051] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-931663746-172.31.14.131-1693151986863 (Datanode Uuid fd80fdff-7dc0-425b-ae63-fbd550d276e3) service to localhost/127.0.0.1:43051 2023-08-27 16:01:46,177 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/cluster_270e8e64-5914-4b90-10eb-e150c3ad6967/dfs/data/data3/current/BP-931663746-172.31.14.131-1693151986863] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 16:01:46,178 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/cluster_270e8e64-5914-4b90-10eb-e150c3ad6967/dfs/data/data4/current/BP-931663746-172.31.14.131-1693151986863] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 16:01:46,179 WARN [Listener at localhost/34587] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-27 16:01:46,182 INFO [Listener at localhost/34587] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 16:01:46,286 WARN [BP-931663746-172.31.14.131-1693151986863 heartbeating to localhost/127.0.0.1:43051] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-27 16:01:46,286 WARN [BP-931663746-172.31.14.131-1693151986863 heartbeating to localhost/127.0.0.1:43051] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-931663746-172.31.14.131-1693151986863 (Datanode Uuid a6217e0f-3e35-41c2-a839-0de9a7ffe5aa) service to localhost/127.0.0.1:43051 2023-08-27 16:01:46,286 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/cluster_270e8e64-5914-4b90-10eb-e150c3ad6967/dfs/data/data1/current/BP-931663746-172.31.14.131-1693151986863] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 16:01:46,287 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/cluster_270e8e64-5914-4b90-10eb-e150c3ad6967/dfs/data/data2/current/BP-931663746-172.31.14.131-1693151986863] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 16:01:46,298 INFO [Listener at localhost/34587] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 16:01:46,414 INFO [Listener at localhost/34587] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-27 16:01:46,446 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-27 16:01:46,455 INFO [Listener at localhost/34587] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=75 (was 64) - Thread LEAK? -, OpenFileDescriptor=463 (was 438) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=24 (was 128), ProcessCount=167 (was 167), AvailableMemoryMB=12559 (was 12713) 2023-08-27 16:01:46,463 INFO [Listener at localhost/34587] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=76, OpenFileDescriptor=463, MaxFileDescriptor=60000, SystemLoadAverage=24, ProcessCount=167, AvailableMemoryMB=12558 2023-08-27 16:01:46,463 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-27 16:01:46,463 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/hadoop.log.dir so I do NOT create it in target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9 2023-08-27 16:01:46,463 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0f1954c-584e-62f5-c708-8e1a269a629c/hadoop.tmp.dir so I do NOT create it in target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9 2023-08-27 16:01:46,463 INFO [Listener at localhost/34587] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/cluster_6a1d2163-f9b0-ef59-34ca-1dac95c3fbf8, deleteOnExit=true 2023-08-27 16:01:46,463 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-27 16:01:46,464 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/test.cache.data in system properties and HBase conf 2023-08-27 16:01:46,464 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/hadoop.tmp.dir in system properties and HBase conf 2023-08-27 16:01:46,464 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/hadoop.log.dir in system properties and HBase conf 2023-08-27 16:01:46,464 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-27 16:01:46,464 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-27 16:01:46,464 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-27 16:01:46,464 DEBUG [Listener at localhost/34587] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-27 16:01:46,464 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-27 16:01:46,465 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-27 16:01:46,465 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-27 16:01:46,465 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-27 16:01:46,465 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-27 16:01:46,465 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-27 16:01:46,465 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-27 16:01:46,465 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-27 16:01:46,465 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-27 16:01:46,465 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/nfs.dump.dir in system properties and HBase conf 2023-08-27 16:01:46,465 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/java.io.tmpdir in system properties and HBase conf 2023-08-27 16:01:46,465 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-27 16:01:46,466 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-27 16:01:46,466 INFO [Listener at localhost/34587] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-27 16:01:46,470 WARN [Listener at localhost/34587] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-27 16:01:46,470 WARN [Listener at localhost/34587] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-27 16:01:46,507 WARN [Listener at localhost/34587] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 16:01:46,509 INFO [Listener at localhost/34587] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 16:01:46,515 INFO [Listener at localhost/34587] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/java.io.tmpdir/Jetty_localhost_42293_hdfs____ciuczc/webapp 2023-08-27 16:01:46,611 INFO [Listener at localhost/34587] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42293 2023-08-27 16:01:46,615 WARN [Listener at localhost/34587] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-27 16:01:46,615 WARN [Listener at localhost/34587] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-27 16:01:46,650 WARN [Listener at localhost/34925] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 16:01:46,660 WARN [Listener at localhost/34925] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-27 16:01:46,662 WARN [Listener at localhost/34925] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 16:01:46,663 INFO [Listener at localhost/34925] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 16:01:46,668 INFO [Listener at localhost/34925] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/java.io.tmpdir/Jetty_localhost_40573_datanode____.6bbttw/webapp 2023-08-27 16:01:46,761 INFO [Listener at localhost/34925] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40573 2023-08-27 16:01:46,766 WARN [Listener at localhost/37153] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 16:01:46,779 WARN [Listener at localhost/37153] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-27 16:01:46,781 WARN [Listener at localhost/37153] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-27 16:01:46,783 INFO [Listener at localhost/37153] log.Slf4jLog(67): jetty-6.1.26 2023-08-27 16:01:46,787 INFO [Listener at localhost/37153] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/java.io.tmpdir/Jetty_localhost_36007_datanode____.mu4qp1/webapp 2023-08-27 16:01:46,857 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x16609251851b27e1: Processing first storage report for DS-82b059d9-4610-4961-969c-59928db7b05e from datanode 71f7b12d-3ad6-4396-9cd6-bb4a9f905c94 2023-08-27 16:01:46,857 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x16609251851b27e1: from storage DS-82b059d9-4610-4961-969c-59928db7b05e node DatanodeRegistration(127.0.0.1:38003, datanodeUuid=71f7b12d-3ad6-4396-9cd6-bb4a9f905c94, infoPort=40197, infoSecurePort=0, ipcPort=37153, storageInfo=lv=-57;cid=testClusterID;nsid=1715495308;c=1693152106472), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 16:01:46,857 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x16609251851b27e1: Processing first storage report for DS-a60a5419-636b-42d8-934f-2744dc8bea84 from datanode 71f7b12d-3ad6-4396-9cd6-bb4a9f905c94 2023-08-27 16:01:46,857 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x16609251851b27e1: from storage DS-a60a5419-636b-42d8-934f-2744dc8bea84 node DatanodeRegistration(127.0.0.1:38003, datanodeUuid=71f7b12d-3ad6-4396-9cd6-bb4a9f905c94, infoPort=40197, infoSecurePort=0, ipcPort=37153, storageInfo=lv=-57;cid=testClusterID;nsid=1715495308;c=1693152106472), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 16:01:46,884 INFO [Listener at localhost/37153] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36007 2023-08-27 16:01:46,890 WARN [Listener at localhost/37897] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-27 16:01:46,980 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x2237c092ccecd63c: Processing first storage report for DS-535f4799-9066-4a70-9795-ce23e00be9b0 from datanode 6e5976c8-f44a-46c8-989a-1accf7b24b38 2023-08-27 16:01:46,981 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x2237c092ccecd63c: from storage DS-535f4799-9066-4a70-9795-ce23e00be9b0 node DatanodeRegistration(127.0.0.1:40377, datanodeUuid=6e5976c8-f44a-46c8-989a-1accf7b24b38, infoPort=33861, infoSecurePort=0, ipcPort=37897, storageInfo=lv=-57;cid=testClusterID;nsid=1715495308;c=1693152106472), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 16:01:46,981 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x2237c092ccecd63c: Processing first storage report for DS-2a572e6f-f9e9-445f-8911-e4d0254d2470 from datanode 6e5976c8-f44a-46c8-989a-1accf7b24b38 2023-08-27 16:01:46,981 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x2237c092ccecd63c: from storage DS-2a572e6f-f9e9-445f-8911-e4d0254d2470 node DatanodeRegistration(127.0.0.1:40377, datanodeUuid=6e5976c8-f44a-46c8-989a-1accf7b24b38, infoPort=33861, infoSecurePort=0, ipcPort=37897, storageInfo=lv=-57;cid=testClusterID;nsid=1715495308;c=1693152106472), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-27 16:01:46,995 DEBUG [Listener at localhost/37897] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9 2023-08-27 16:01:46,997 INFO [Listener at localhost/37897] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/cluster_6a1d2163-f9b0-ef59-34ca-1dac95c3fbf8/zookeeper_0, clientPort=65109, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/cluster_6a1d2163-f9b0-ef59-34ca-1dac95c3fbf8/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/cluster_6a1d2163-f9b0-ef59-34ca-1dac95c3fbf8/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-27 16:01:46,998 INFO [Listener at localhost/37897] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=65109 2023-08-27 16:01:46,999 INFO [Listener at localhost/37897] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 16:01:46,999 INFO [Listener at localhost/37897] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 16:01:47,011 INFO [Listener at localhost/37897] util.FSUtils(471): Created version file at hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee with version=8 2023-08-27 16:01:47,011 INFO [Listener at localhost/37897] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:46535/user/jenkins/test-data/8f68a77a-0354-2174-b589-2a20441d7fc2/hbase-staging 2023-08-27 16:01:47,013 INFO [Listener at localhost/37897] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-27 16:01:47,013 INFO [Listener at localhost/37897] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 16:01:47,013 INFO [Listener at localhost/37897] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-27 16:01:47,013 INFO [Listener at localhost/37897] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-27 16:01:47,013 INFO [Listener at localhost/37897] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 16:01:47,013 INFO [Listener at localhost/37897] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-27 16:01:47,014 INFO [Listener at localhost/37897] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-27 16:01:47,015 INFO [Listener at localhost/37897] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46313 2023-08-27 16:01:47,015 INFO [Listener at localhost/37897] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 16:01:47,016 INFO [Listener at localhost/37897] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 16:01:47,017 INFO [Listener at localhost/37897] zookeeper.RecoverableZooKeeper(93): Process identifier=master:46313 connecting to ZooKeeper ensemble=127.0.0.1:65109 2023-08-27 16:01:47,025 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:463130x0, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-27 16:01:47,026 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:46313-0x1004e7a119b0000 connected 2023-08-27 16:01:47,043 DEBUG [Listener at localhost/37897] zookeeper.ZKUtil(164): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-27 16:01:47,043 DEBUG [Listener at localhost/37897] zookeeper.ZKUtil(164): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 16:01:47,044 DEBUG [Listener at localhost/37897] zookeeper.ZKUtil(164): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-27 16:01:47,044 DEBUG [Listener at localhost/37897] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46313 2023-08-27 16:01:47,044 DEBUG [Listener at localhost/37897] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46313 2023-08-27 16:01:47,045 DEBUG [Listener at localhost/37897] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46313 2023-08-27 16:01:47,047 DEBUG [Listener at localhost/37897] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46313 2023-08-27 16:01:47,048 DEBUG [Listener at localhost/37897] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46313 2023-08-27 16:01:47,048 INFO [Listener at localhost/37897] master.HMaster(444): hbase.rootdir=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee, hbase.cluster.distributed=false 2023-08-27 16:01:47,060 INFO [Listener at localhost/37897] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-27 16:01:47,060 INFO [Listener at localhost/37897] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 16:01:47,061 INFO [Listener at localhost/37897] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-27 16:01:47,061 INFO [Listener at localhost/37897] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-27 16:01:47,061 INFO [Listener at localhost/37897] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-27 16:01:47,061 INFO [Listener at localhost/37897] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-27 16:01:47,061 INFO [Listener at localhost/37897] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-27 16:01:47,063 INFO [Listener at localhost/37897] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46435 2023-08-27 16:01:47,063 INFO [Listener at localhost/37897] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-27 16:01:47,064 DEBUG [Listener at localhost/37897] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-27 16:01:47,064 INFO [Listener at localhost/37897] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 16:01:47,065 INFO [Listener at localhost/37897] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 16:01:47,066 INFO [Listener at localhost/37897] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46435 connecting to ZooKeeper ensemble=127.0.0.1:65109 2023-08-27 16:01:47,069 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): regionserver:464350x0, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-27 16:01:47,070 DEBUG [Listener at localhost/37897] zookeeper.ZKUtil(164): regionserver:464350x0, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-27 16:01:47,070 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46435-0x1004e7a119b0001 connected 2023-08-27 16:01:47,071 DEBUG [Listener at localhost/37897] zookeeper.ZKUtil(164): regionserver:46435-0x1004e7a119b0001, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 16:01:47,072 DEBUG [Listener at localhost/37897] zookeeper.ZKUtil(164): regionserver:46435-0x1004e7a119b0001, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-27 16:01:47,072 DEBUG [Listener at localhost/37897] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46435 2023-08-27 16:01:47,073 DEBUG [Listener at localhost/37897] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46435 2023-08-27 16:01:47,073 DEBUG [Listener at localhost/37897] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46435 2023-08-27 16:01:47,077 DEBUG [Listener at localhost/37897] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46435 2023-08-27 16:01:47,077 DEBUG [Listener at localhost/37897] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46435 2023-08-27 16:01:47,081 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,46313,1693152107013 2023-08-27 16:01:47,083 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-27 16:01:47,083 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,46313,1693152107013 2023-08-27 16:01:47,085 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): regionserver:46435-0x1004e7a119b0001, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-27 16:01:47,085 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-27 16:01:47,085 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 16:01:47,085 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-27 16:01:47,086 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,46313,1693152107013 from backup master directory 2023-08-27 16:01:47,086 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-27 16:01:47,088 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,46313,1693152107013 2023-08-27 16:01:47,088 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-27 16:01:47,088 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-27 16:01:47,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,46313,1693152107013 2023-08-27 16:01:47,100 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/hbase.id with ID: 935f5a38-7805-4631-9554-f100de0776f0 2023-08-27 16:01:47,110 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 16:01:47,112 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 16:01:47,119 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x4b7be0cd to 127.0.0.1:65109 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-27 16:01:47,122 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@76db8f3b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-27 16:01:47,122 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-27 16:01:47,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-27 16:01:47,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-27 16:01:47,124 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/MasterData/data/master/store-tmp 2023-08-27 16:01:47,131 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 16:01:47,131 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-27 16:01:47,131 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 16:01:47,131 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 16:01:47,131 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-27 16:01:47,131 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 16:01:47,131 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 16:01:47,131 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-27 16:01:47,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/MasterData/WALs/jenkins-hbase4.apache.org,46313,1693152107013 2023-08-27 16:01:47,134 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46313%2C1693152107013, suffix=, logDir=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/MasterData/WALs/jenkins-hbase4.apache.org,46313,1693152107013, archiveDir=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/MasterData/oldWALs, maxLogs=10 2023-08-27 16:01:47,148 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40377,DS-535f4799-9066-4a70-9795-ce23e00be9b0,DISK] 2023-08-27 16:01:47,148 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38003,DS-82b059d9-4610-4961-969c-59928db7b05e,DISK] 2023-08-27 16:01:47,150 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/MasterData/WALs/jenkins-hbase4.apache.org,46313,1693152107013/jenkins-hbase4.apache.org%2C46313%2C1693152107013.1693152107134 2023-08-27 16:01:47,150 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40377,DS-535f4799-9066-4a70-9795-ce23e00be9b0,DISK], DatanodeInfoWithStorage[127.0.0.1:38003,DS-82b059d9-4610-4961-969c-59928db7b05e,DISK]] 2023-08-27 16:01:47,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-27 16:01:47,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 16:01:47,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-27 16:01:47,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-27 16:01:47,152 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-27 16:01:47,154 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-27 16:01:47,154 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-27 16:01:47,154 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 16:01:47,155 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-27 16:01:47,156 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-27 16:01:47,158 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-27 16:01:47,160 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-27 16:01:47,160 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=784110, jitterRate=-0.0029534995555877686}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-27 16:01:47,160 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-27 16:01:47,160 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-27 16:01:47,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-27 16:01:47,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-27 16:01:47,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-27 16:01:47,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-27 16:01:47,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-08-27 16:01:47,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-27 16:01:47,163 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-27 16:01:47,163 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-27 16:01:47,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-27 16:01:47,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-27 16:01:47,174 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-27 16:01:47,174 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-27 16:01:47,174 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-27 16:01:47,176 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 16:01:47,177 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-27 16:01:47,177 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-27 16:01:47,178 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-27 16:01:47,180 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-27 16:01:47,180 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): regionserver:46435-0x1004e7a119b0001, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-27 16:01:47,180 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 16:01:47,181 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,46313,1693152107013, sessionid=0x1004e7a119b0000, setting cluster-up flag (Was=false) 2023-08-27 16:01:47,184 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 16:01:47,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-27 16:01:47,189 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46313,1693152107013 2023-08-27 16:01:47,192 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 16:01:47,197 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-27 16:01:47,197 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46313,1693152107013 2023-08-27 16:01:47,198 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/.hbase-snapshot/.tmp 2023-08-27 16:01:47,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-27 16:01:47,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 16:01:47,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 16:01:47,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 16:01:47,200 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-27 16:01:47,200 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-27 16:01:47,200 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 16:01:47,200 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-27 16:01:47,200 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 16:01:47,201 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693152137201 2023-08-27 16:01:47,201 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-27 16:01:47,201 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-27 16:01:47,201 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-27 16:01:47,201 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-27 16:01:47,201 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-27 16:01:47,201 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-27 16:01:47,201 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-27 16:01:47,202 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-27 16:01:47,202 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-27 16:01:47,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-27 16:01:47,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-27 16:01:47,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-27 16:01:47,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-27 16:01:47,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-27 16:01:47,202 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693152107202,5,FailOnTimeoutGroup] 2023-08-27 16:01:47,203 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693152107203,5,FailOnTimeoutGroup] 2023-08-27 16:01:47,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-27 16:01:47,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-27 16:01:47,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-27 16:01:47,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-27 16:01:47,203 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-27 16:01:47,211 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-27 16:01:47,211 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-27 16:01:47,211 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee 2023-08-27 16:01:47,218 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 16:01:47,219 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-27 16:01:47,221 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/info 2023-08-27 16:01:47,221 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-27 16:01:47,222 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 16:01:47,222 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-27 16:01:47,223 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/rep_barrier 2023-08-27 16:01:47,223 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-27 16:01:47,223 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 16:01:47,224 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-27 16:01:47,225 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/table 2023-08-27 16:01:47,225 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-27 16:01:47,225 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 16:01:47,226 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740 2023-08-27 16:01:47,227 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740 2023-08-27 16:01:47,228 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-27 16:01:47,230 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-27 16:01:47,231 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-27 16:01:47,232 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=773680, jitterRate=-0.01621551811695099}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-27 16:01:47,232 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-27 16:01:47,232 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-27 16:01:47,232 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-27 16:01:47,232 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-27 16:01:47,232 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-27 16:01:47,232 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-27 16:01:47,232 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-27 16:01:47,232 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-27 16:01:47,233 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-27 16:01:47,233 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-27 16:01:47,233 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-27 16:01:47,234 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-27 16:01:47,235 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-27 16:01:47,282 INFO [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(951): ClusterId : 935f5a38-7805-4631-9554-f100de0776f0 2023-08-27 16:01:47,282 DEBUG [RS:0;jenkins-hbase4:46435] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-27 16:01:47,284 DEBUG [RS:0;jenkins-hbase4:46435] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-27 16:01:47,284 DEBUG [RS:0;jenkins-hbase4:46435] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-27 16:01:47,292 DEBUG [RS:0;jenkins-hbase4:46435] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-27 16:01:47,293 DEBUG [RS:0;jenkins-hbase4:46435] zookeeper.ReadOnlyZKClient(139): Connect 0x2e8e7be7 to 127.0.0.1:65109 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-27 16:01:47,296 DEBUG [RS:0;jenkins-hbase4:46435] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@20498046, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-27 16:01:47,296 DEBUG [RS:0;jenkins-hbase4:46435] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@24761097, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-27 16:01:47,305 DEBUG [RS:0;jenkins-hbase4:46435] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:46435 2023-08-27 16:01:47,305 INFO [RS:0;jenkins-hbase4:46435] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-27 16:01:47,305 INFO [RS:0;jenkins-hbase4:46435] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-27 16:01:47,305 DEBUG [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(1022): About to register with Master. 2023-08-27 16:01:47,305 INFO [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,46313,1693152107013 with isa=jenkins-hbase4.apache.org/172.31.14.131:46435, startcode=1693152107060 2023-08-27 16:01:47,305 DEBUG [RS:0;jenkins-hbase4:46435] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-27 16:01:47,307 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47697, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-08-27 16:01:47,308 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=46313] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46435,1693152107060 2023-08-27 16:01:47,309 DEBUG [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee 2023-08-27 16:01:47,309 DEBUG [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:34925 2023-08-27 16:01:47,309 DEBUG [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-27 16:01:47,310 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-27 16:01:47,310 DEBUG [RS:0;jenkins-hbase4:46435] zookeeper.ZKUtil(162): regionserver:46435-0x1004e7a119b0001, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46435,1693152107060 2023-08-27 16:01:47,310 WARN [RS:0;jenkins-hbase4:46435] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-27 16:01:47,311 INFO [RS:0;jenkins-hbase4:46435] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-27 16:01:47,311 DEBUG [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(1948): logDir=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/WALs/jenkins-hbase4.apache.org,46435,1693152107060 2023-08-27 16:01:47,311 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46435,1693152107060] 2023-08-27 16:01:47,315 DEBUG [RS:0;jenkins-hbase4:46435] zookeeper.ZKUtil(162): regionserver:46435-0x1004e7a119b0001, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46435,1693152107060 2023-08-27 16:01:47,316 DEBUG [RS:0;jenkins-hbase4:46435] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-27 16:01:47,316 INFO [RS:0;jenkins-hbase4:46435] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-27 16:01:47,317 INFO [RS:0;jenkins-hbase4:46435] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-27 16:01:47,317 INFO [RS:0;jenkins-hbase4:46435] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-27 16:01:47,317 INFO [RS:0;jenkins-hbase4:46435] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-27 16:01:47,317 INFO [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-27 16:01:47,319 INFO [RS:0;jenkins-hbase4:46435] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-27 16:01:47,319 DEBUG [RS:0;jenkins-hbase4:46435] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 16:01:47,320 DEBUG [RS:0;jenkins-hbase4:46435] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 16:01:47,320 DEBUG [RS:0;jenkins-hbase4:46435] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 16:01:47,320 DEBUG [RS:0;jenkins-hbase4:46435] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 16:01:47,320 DEBUG [RS:0;jenkins-hbase4:46435] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 16:01:47,320 DEBUG [RS:0;jenkins-hbase4:46435] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-27 16:01:47,320 DEBUG [RS:0;jenkins-hbase4:46435] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 16:01:47,320 DEBUG [RS:0;jenkins-hbase4:46435] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 16:01:47,320 DEBUG [RS:0;jenkins-hbase4:46435] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 16:01:47,320 DEBUG [RS:0;jenkins-hbase4:46435] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-27 16:01:47,321 INFO [RS:0;jenkins-hbase4:46435] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-27 16:01:47,321 INFO [RS:0;jenkins-hbase4:46435] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-27 16:01:47,321 INFO [RS:0;jenkins-hbase4:46435] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-27 16:01:47,334 INFO [RS:0;jenkins-hbase4:46435] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-27 16:01:47,334 INFO [RS:0;jenkins-hbase4:46435] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46435,1693152107060-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-27 16:01:47,344 INFO [RS:0;jenkins-hbase4:46435] regionserver.Replication(203): jenkins-hbase4.apache.org,46435,1693152107060 started 2023-08-27 16:01:47,344 INFO [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46435,1693152107060, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46435, sessionid=0x1004e7a119b0001 2023-08-27 16:01:47,344 DEBUG [RS:0;jenkins-hbase4:46435] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-27 16:01:47,344 DEBUG [RS:0;jenkins-hbase4:46435] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46435,1693152107060 2023-08-27 16:01:47,344 DEBUG [RS:0;jenkins-hbase4:46435] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46435,1693152107060' 2023-08-27 16:01:47,344 DEBUG [RS:0;jenkins-hbase4:46435] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-27 16:01:47,345 DEBUG [RS:0;jenkins-hbase4:46435] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-27 16:01:47,345 DEBUG [RS:0;jenkins-hbase4:46435] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-27 16:01:47,345 DEBUG [RS:0;jenkins-hbase4:46435] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-27 16:01:47,345 DEBUG [RS:0;jenkins-hbase4:46435] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46435,1693152107060 2023-08-27 16:01:47,345 DEBUG [RS:0;jenkins-hbase4:46435] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46435,1693152107060' 2023-08-27 16:01:47,345 DEBUG [RS:0;jenkins-hbase4:46435] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-27 16:01:47,345 DEBUG [RS:0;jenkins-hbase4:46435] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-27 16:01:47,346 DEBUG [RS:0;jenkins-hbase4:46435] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-27 16:01:47,346 INFO [RS:0;jenkins-hbase4:46435] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-27 16:01:47,346 INFO [RS:0;jenkins-hbase4:46435] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-27 16:01:47,385 DEBUG [jenkins-hbase4:46313] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-27 16:01:47,386 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46435,1693152107060, state=OPENING 2023-08-27 16:01:47,387 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-27 16:01:47,390 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 16:01:47,390 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-27 16:01:47,390 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46435,1693152107060}] 2023-08-27 16:01:47,447 INFO [RS:0;jenkins-hbase4:46435] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46435%2C1693152107060, suffix=, logDir=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/WALs/jenkins-hbase4.apache.org,46435,1693152107060, archiveDir=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/oldWALs, maxLogs=32 2023-08-27 16:01:47,466 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38003,DS-82b059d9-4610-4961-969c-59928db7b05e,DISK] 2023-08-27 16:01:47,466 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40377,DS-535f4799-9066-4a70-9795-ce23e00be9b0,DISK] 2023-08-27 16:01:47,467 INFO [RS:0;jenkins-hbase4:46435] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/WALs/jenkins-hbase4.apache.org,46435,1693152107060/jenkins-hbase4.apache.org%2C46435%2C1693152107060.1693152107448 2023-08-27 16:01:47,468 DEBUG [RS:0;jenkins-hbase4:46435] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38003,DS-82b059d9-4610-4961-969c-59928db7b05e,DISK], DatanodeInfoWithStorage[127.0.0.1:40377,DS-535f4799-9066-4a70-9795-ce23e00be9b0,DISK]] 2023-08-27 16:01:47,544 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46435,1693152107060 2023-08-27 16:01:47,544 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-27 16:01:47,546 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60950, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-27 16:01:47,549 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-27 16:01:47,549 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-27 16:01:47,550 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46435%2C1693152107060.meta, suffix=.meta, logDir=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/WALs/jenkins-hbase4.apache.org,46435,1693152107060, archiveDir=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/oldWALs, maxLogs=32 2023-08-27 16:01:47,564 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38003,DS-82b059d9-4610-4961-969c-59928db7b05e,DISK] 2023-08-27 16:01:47,564 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40377,DS-535f4799-9066-4a70-9795-ce23e00be9b0,DISK] 2023-08-27 16:01:47,565 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/WALs/jenkins-hbase4.apache.org,46435,1693152107060/jenkins-hbase4.apache.org%2C46435%2C1693152107060.meta.1693152107551.meta 2023-08-27 16:01:47,566 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40377,DS-535f4799-9066-4a70-9795-ce23e00be9b0,DISK], DatanodeInfoWithStorage[127.0.0.1:38003,DS-82b059d9-4610-4961-969c-59928db7b05e,DISK]] 2023-08-27 16:01:47,566 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-27 16:01:47,566 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-27 16:01:47,566 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-27 16:01:47,566 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-27 16:01:47,566 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-27 16:01:47,566 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 16:01:47,566 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-27 16:01:47,566 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-27 16:01:47,567 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-27 16:01:47,568 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/info 2023-08-27 16:01:47,568 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/info 2023-08-27 16:01:47,569 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-27 16:01:47,569 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 16:01:47,569 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-27 16:01:47,570 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/rep_barrier 2023-08-27 16:01:47,570 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/rep_barrier 2023-08-27 16:01:47,570 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-27 16:01:47,571 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 16:01:47,571 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-27 16:01:47,572 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/table 2023-08-27 16:01:47,572 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/table 2023-08-27 16:01:47,572 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-27 16:01:47,572 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 16:01:47,573 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740 2023-08-27 16:01:47,574 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740 2023-08-27 16:01:47,576 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-27 16:01:47,577 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-27 16:01:47,578 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=815558, jitterRate=0.037035852670669556}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-27 16:01:47,578 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-27 16:01:47,579 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693152107544 2023-08-27 16:01:47,582 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-27 16:01:47,583 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-27 16:01:47,583 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46435,1693152107060, state=OPEN 2023-08-27 16:01:47,585 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-27 16:01:47,585 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-27 16:01:47,586 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-27 16:01:47,586 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46435,1693152107060 in 195 msec 2023-08-27 16:01:47,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-27 16:01:47,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 353 msec 2023-08-27 16:01:47,589 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 390 msec 2023-08-27 16:01:47,589 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693152107589, completionTime=-1 2023-08-27 16:01:47,589 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-27 16:01:47,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-27 16:01:47,591 DEBUG [hconnection-0x5d0a0f7d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-27 16:01:47,593 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60954, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-27 16:01:47,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-27 16:01:47,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693152167594 2023-08-27 16:01:47,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693152227594 2023-08-27 16:01:47,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-27 16:01:47,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46313,1693152107013-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-27 16:01:47,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46313,1693152107013-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-27 16:01:47,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46313,1693152107013-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-27 16:01:47,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:46313, period=300000, unit=MILLISECONDS is enabled. 2023-08-27 16:01:47,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-27 16:01:47,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-27 16:01:47,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-27 16:01:47,603 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-27 16:01:47,604 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-27 16:01:47,604 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-27 16:01:47,605 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-27 16:01:47,606 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/.tmp/data/hbase/namespace/f8cc749cc9151a9c9d2eb20ccc756982 2023-08-27 16:01:47,606 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/.tmp/data/hbase/namespace/f8cc749cc9151a9c9d2eb20ccc756982 empty. 2023-08-27 16:01:47,607 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/.tmp/data/hbase/namespace/f8cc749cc9151a9c9d2eb20ccc756982 2023-08-27 16:01:47,607 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-27 16:01:47,616 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-27 16:01:47,617 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => f8cc749cc9151a9c9d2eb20ccc756982, NAME => 'hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/.tmp 2023-08-27 16:01:47,624 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 16:01:47,624 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing f8cc749cc9151a9c9d2eb20ccc756982, disabling compactions & flushes 2023-08-27 16:01:47,625 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982. 2023-08-27 16:01:47,625 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982. 2023-08-27 16:01:47,625 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982. after waiting 0 ms 2023-08-27 16:01:47,625 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982. 2023-08-27 16:01:47,625 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982. 2023-08-27 16:01:47,625 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for f8cc749cc9151a9c9d2eb20ccc756982: 2023-08-27 16:01:47,627 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-27 16:01:47,628 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693152107628"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693152107628"}]},"ts":"1693152107628"} 2023-08-27 16:01:47,629 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-27 16:01:47,630 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-27 16:01:47,630 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693152107630"}]},"ts":"1693152107630"} 2023-08-27 16:01:47,631 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-27 16:01:47,638 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f8cc749cc9151a9c9d2eb20ccc756982, ASSIGN}] 2023-08-27 16:01:47,639 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f8cc749cc9151a9c9d2eb20ccc756982, ASSIGN 2023-08-27 16:01:47,640 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=f8cc749cc9151a9c9d2eb20ccc756982, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46435,1693152107060; forceNewPlan=false, retain=false 2023-08-27 16:01:47,791 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f8cc749cc9151a9c9d2eb20ccc756982, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46435,1693152107060 2023-08-27 16:01:47,791 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693152107791"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693152107791"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693152107791"}]},"ts":"1693152107791"} 2023-08-27 16:01:47,793 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure f8cc749cc9151a9c9d2eb20ccc756982, server=jenkins-hbase4.apache.org,46435,1693152107060}] 2023-08-27 16:01:47,948 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982. 2023-08-27 16:01:47,949 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f8cc749cc9151a9c9d2eb20ccc756982, NAME => 'hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982.', STARTKEY => '', ENDKEY => ''} 2023-08-27 16:01:47,949 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace f8cc749cc9151a9c9d2eb20ccc756982 2023-08-27 16:01:47,949 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-27 16:01:47,949 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f8cc749cc9151a9c9d2eb20ccc756982 2023-08-27 16:01:47,949 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f8cc749cc9151a9c9d2eb20ccc756982 2023-08-27 16:01:47,950 INFO [StoreOpener-f8cc749cc9151a9c9d2eb20ccc756982-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region f8cc749cc9151a9c9d2eb20ccc756982 2023-08-27 16:01:47,952 DEBUG [StoreOpener-f8cc749cc9151a9c9d2eb20ccc756982-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/namespace/f8cc749cc9151a9c9d2eb20ccc756982/info 2023-08-27 16:01:47,952 DEBUG [StoreOpener-f8cc749cc9151a9c9d2eb20ccc756982-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/namespace/f8cc749cc9151a9c9d2eb20ccc756982/info 2023-08-27 16:01:47,952 INFO [StoreOpener-f8cc749cc9151a9c9d2eb20ccc756982-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f8cc749cc9151a9c9d2eb20ccc756982 columnFamilyName info 2023-08-27 16:01:47,952 INFO [StoreOpener-f8cc749cc9151a9c9d2eb20ccc756982-1] regionserver.HStore(310): Store=f8cc749cc9151a9c9d2eb20ccc756982/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-27 16:01:47,953 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/namespace/f8cc749cc9151a9c9d2eb20ccc756982 2023-08-27 16:01:47,955 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/namespace/f8cc749cc9151a9c9d2eb20ccc756982 2023-08-27 16:01:47,958 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f8cc749cc9151a9c9d2eb20ccc756982 2023-08-27 16:01:47,959 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/namespace/f8cc749cc9151a9c9d2eb20ccc756982/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-27 16:01:47,960 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f8cc749cc9151a9c9d2eb20ccc756982; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=821364, jitterRate=0.044419094920158386}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-27 16:01:47,960 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f8cc749cc9151a9c9d2eb20ccc756982: 2023-08-27 16:01:47,961 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982., pid=6, masterSystemTime=1693152107945 2023-08-27 16:01:47,962 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982. 2023-08-27 16:01:47,962 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982. 2023-08-27 16:01:47,962 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f8cc749cc9151a9c9d2eb20ccc756982, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46435,1693152107060 2023-08-27 16:01:47,963 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693152107962"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693152107962"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693152107962"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693152107962"}]},"ts":"1693152107962"} 2023-08-27 16:01:47,965 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-27 16:01:47,965 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure f8cc749cc9151a9c9d2eb20ccc756982, server=jenkins-hbase4.apache.org,46435,1693152107060 in 171 msec 2023-08-27 16:01:47,967 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-27 16:01:47,967 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=f8cc749cc9151a9c9d2eb20ccc756982, ASSIGN in 327 msec 2023-08-27 16:01:47,968 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-27 16:01:47,968 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693152107968"}]},"ts":"1693152107968"} 2023-08-27 16:01:47,969 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-27 16:01:47,971 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-27 16:01:47,973 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 369 msec 2023-08-27 16:01:48,004 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-27 16:01:48,005 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-27 16:01:48,005 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 16:01:48,009 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-27 16:01:48,017 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-27 16:01:48,019 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 10 msec 2023-08-27 16:01:48,020 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-27 16:01:48,025 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-27 16:01:48,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 7 msec 2023-08-27 16:01:48,034 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-27 16:01:48,037 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-27 16:01:48,037 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 0.949sec 2023-08-27 16:01:48,037 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-27 16:01:48,037 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-27 16:01:48,037 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-27 16:01:48,037 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46313,1693152107013-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-27 16:01:48,037 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46313,1693152107013-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-27 16:01:48,039 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-27 16:01:48,082 DEBUG [Listener at localhost/37897] zookeeper.ReadOnlyZKClient(139): Connect 0x2d6a62f6 to 127.0.0.1:65109 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-27 16:01:48,088 DEBUG [Listener at localhost/37897] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@62d835e8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-27 16:01:48,091 DEBUG [hconnection-0x3a26c626-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-27 16:01:48,093 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60956, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-27 16:01:48,094 INFO [Listener at localhost/37897] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,46313,1693152107013 2023-08-27 16:01:48,094 INFO [Listener at localhost/37897] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-27 16:01:48,098 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-27 16:01:48,098 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 16:01:48,099 INFO [Listener at localhost/37897] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-27 16:01:48,099 INFO [Listener at localhost/37897] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-27 16:01:48,101 INFO [Listener at localhost/37897] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=test.com%2C8080%2C1, suffix=, logDir=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/WALs/test.com,8080,1, archiveDir=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/oldWALs, maxLogs=32 2023-08-27 16:01:48,115 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38003,DS-82b059d9-4610-4961-969c-59928db7b05e,DISK] 2023-08-27 16:01:48,116 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40377,DS-535f4799-9066-4a70-9795-ce23e00be9b0,DISK] 2023-08-27 16:01:48,117 INFO [Listener at localhost/37897] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/WALs/test.com,8080,1/test.com%2C8080%2C1.1693152108101 2023-08-27 16:01:48,118 DEBUG [Listener at localhost/37897] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38003,DS-82b059d9-4610-4961-969c-59928db7b05e,DISK], DatanodeInfoWithStorage[127.0.0.1:40377,DS-535f4799-9066-4a70-9795-ce23e00be9b0,DISK]] 2023-08-27 16:01:48,133 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40377,DS-535f4799-9066-4a70-9795-ce23e00be9b0,DISK] 2023-08-27 16:01:48,133 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38003,DS-82b059d9-4610-4961-969c-59928db7b05e,DISK] 2023-08-27 16:01:48,135 INFO [Listener at localhost/37897] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/WALs/test.com,8080,1/test.com%2C8080%2C1.1693152108101 with entries=0, filesize=83 B; new WAL /user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/WALs/test.com,8080,1/test.com%2C8080%2C1.1693152108118 2023-08-27 16:01:48,135 DEBUG [Listener at localhost/37897] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40377,DS-535f4799-9066-4a70-9795-ce23e00be9b0,DISK], DatanodeInfoWithStorage[127.0.0.1:38003,DS-82b059d9-4610-4961-969c-59928db7b05e,DISK]] 2023-08-27 16:01:48,135 DEBUG [Listener at localhost/37897] wal.AbstractFSWAL(716): hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/WALs/test.com,8080,1/test.com%2C8080%2C1.1693152108101 is not closed yet, will try archiving it next time 2023-08-27 16:01:48,140 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/WALs/test.com,8080,1/test.com%2C8080%2C1.1693152108101 to hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/oldWALs/test.com%2C8080%2C1.1693152108101 2023-08-27 16:01:48,142 DEBUG [Listener at localhost/37897] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/oldWALs 2023-08-27 16:01:48,143 INFO [Listener at localhost/37897] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL test.com%2C8080%2C1:(num 1693152108118) 2023-08-27 16:01:48,143 INFO [Listener at localhost/37897] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-27 16:01:48,143 DEBUG [Listener at localhost/37897] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2d6a62f6 to 127.0.0.1:65109 2023-08-27 16:01:48,143 DEBUG [Listener at localhost/37897] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 16:01:48,144 DEBUG [Listener at localhost/37897] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-27 16:01:48,144 DEBUG [Listener at localhost/37897] util.JVMClusterUtil(257): Found active master hash=1359393725, stopped=false 2023-08-27 16:01:48,144 INFO [Listener at localhost/37897] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,46313,1693152107013 2023-08-27 16:01:48,146 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-27 16:01:48,146 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): regionserver:46435-0x1004e7a119b0001, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-27 16:01:48,146 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 16:01:48,146 INFO [Listener at localhost/37897] procedure2.ProcedureExecutor(629): Stopping 2023-08-27 16:01:48,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 16:01:48,147 DEBUG [Listener at localhost/37897] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4b7be0cd to 127.0.0.1:65109 2023-08-27 16:01:48,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46435-0x1004e7a119b0001, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-27 16:01:48,148 DEBUG [Listener at localhost/37897] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 16:01:48,149 INFO [Listener at localhost/37897] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46435,1693152107060' ***** 2023-08-27 16:01:48,149 INFO [Listener at localhost/37897] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-27 16:01:48,149 INFO [RS:0;jenkins-hbase4:46435] regionserver.HeapMemoryManager(220): Stopping 2023-08-27 16:01:48,149 INFO [RS:0;jenkins-hbase4:46435] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-27 16:01:48,149 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-27 16:01:48,149 INFO [RS:0;jenkins-hbase4:46435] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-27 16:01:48,150 INFO [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(3305): Received CLOSE for f8cc749cc9151a9c9d2eb20ccc756982 2023-08-27 16:01:48,153 INFO [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46435,1693152107060 2023-08-27 16:01:48,153 DEBUG [RS:0;jenkins-hbase4:46435] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2e8e7be7 to 127.0.0.1:65109 2023-08-27 16:01:48,153 DEBUG [RS:0;jenkins-hbase4:46435] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 16:01:48,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f8cc749cc9151a9c9d2eb20ccc756982, disabling compactions & flushes 2023-08-27 16:01:48,153 INFO [RS:0;jenkins-hbase4:46435] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-27 16:01:48,153 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982. 2023-08-27 16:01:48,153 INFO [RS:0;jenkins-hbase4:46435] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-27 16:01:48,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982. 2023-08-27 16:01:48,153 INFO [RS:0;jenkins-hbase4:46435] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-27 16:01:48,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982. after waiting 0 ms 2023-08-27 16:01:48,154 INFO [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-27 16:01:48,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982. 2023-08-27 16:01:48,154 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing f8cc749cc9151a9c9d2eb20ccc756982 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-27 16:01:48,157 INFO [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-08-27 16:01:48,157 DEBUG [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, f8cc749cc9151a9c9d2eb20ccc756982=hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982.} 2023-08-27 16:01:48,157 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-27 16:01:48,157 DEBUG [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(1504): Waiting on 1588230740, f8cc749cc9151a9c9d2eb20ccc756982 2023-08-27 16:01:48,157 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-27 16:01:48,157 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-27 16:01:48,157 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-27 16:01:48,157 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-27 16:01:48,157 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-08-27 16:01:48,181 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/namespace/f8cc749cc9151a9c9d2eb20ccc756982/.tmp/info/87ecf0be18424951a460e7fbf14de84d 2023-08-27 16:01:48,186 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/.tmp/info/ef2dd3d80fe14acba1a36400e272e756 2023-08-27 16:01:48,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/namespace/f8cc749cc9151a9c9d2eb20ccc756982/.tmp/info/87ecf0be18424951a460e7fbf14de84d as hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/namespace/f8cc749cc9151a9c9d2eb20ccc756982/info/87ecf0be18424951a460e7fbf14de84d 2023-08-27 16:01:48,203 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/namespace/f8cc749cc9151a9c9d2eb20ccc756982/info/87ecf0be18424951a460e7fbf14de84d, entries=2, sequenceid=6, filesize=4.8 K 2023-08-27 16:01:48,205 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for f8cc749cc9151a9c9d2eb20ccc756982 in 51ms, sequenceid=6, compaction requested=false 2023-08-27 16:01:48,205 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-27 16:01:48,210 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/.tmp/table/46c34a3422274d6ea687b0e3edf291c0 2023-08-27 16:01:48,213 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/namespace/f8cc749cc9151a9c9d2eb20ccc756982/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-27 16:01:48,214 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982. 2023-08-27 16:01:48,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f8cc749cc9151a9c9d2eb20ccc756982: 2023-08-27 16:01:48,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693152107602.f8cc749cc9151a9c9d2eb20ccc756982. 2023-08-27 16:01:48,216 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/.tmp/info/ef2dd3d80fe14acba1a36400e272e756 as hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/info/ef2dd3d80fe14acba1a36400e272e756 2023-08-27 16:01:48,220 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/info/ef2dd3d80fe14acba1a36400e272e756, entries=10, sequenceid=9, filesize=5.9 K 2023-08-27 16:01:48,221 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/.tmp/table/46c34a3422274d6ea687b0e3edf291c0 as hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/table/46c34a3422274d6ea687b0e3edf291c0 2023-08-27 16:01:48,226 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/table/46c34a3422274d6ea687b0e3edf291c0, entries=2, sequenceid=9, filesize=4.7 K 2023-08-27 16:01:48,227 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 70ms, sequenceid=9, compaction requested=false 2023-08-27 16:01:48,227 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-27 16:01:48,235 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-08-27 16:01:48,236 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-27 16:01:48,236 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-27 16:01:48,236 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-27 16:01:48,236 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-27 16:01:48,327 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-27 16:01:48,327 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-27 16:01:48,357 INFO [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46435,1693152107060; all regions closed. 2023-08-27 16:01:48,362 DEBUG [RS:0;jenkins-hbase4:46435] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/oldWALs 2023-08-27 16:01:48,362 INFO [RS:0;jenkins-hbase4:46435] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46435%2C1693152107060.meta:.meta(num 1693152107551) 2023-08-27 16:01:48,416 DEBUG [RS:0;jenkins-hbase4:46435] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/oldWALs 2023-08-27 16:01:48,416 INFO [RS:0;jenkins-hbase4:46435] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46435%2C1693152107060:(num 1693152107448) 2023-08-27 16:01:48,416 DEBUG [RS:0;jenkins-hbase4:46435] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 16:01:48,416 INFO [RS:0;jenkins-hbase4:46435] regionserver.LeaseManager(133): Closed leases 2023-08-27 16:01:48,417 INFO [RS:0;jenkins-hbase4:46435] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-27 16:01:48,417 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-27 16:01:48,418 INFO [RS:0;jenkins-hbase4:46435] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46435 2023-08-27 16:01:48,420 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): regionserver:46435-0x1004e7a119b0001, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46435,1693152107060 2023-08-27 16:01:48,420 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-27 16:01:48,420 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): regionserver:46435-0x1004e7a119b0001, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-27 16:01:48,422 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46435,1693152107060] 2023-08-27 16:01:48,422 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46435,1693152107060; numProcessing=1 2023-08-27 16:01:48,423 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,46435,1693152107060 already deleted, retry=false 2023-08-27 16:01:48,424 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46435,1693152107060 expired; onlineServers=0 2023-08-27 16:01:48,424 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46313,1693152107013' ***** 2023-08-27 16:01:48,424 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-27 16:01:48,424 DEBUG [M:0;jenkins-hbase4:46313] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@55392d45, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-27 16:01:48,424 INFO [M:0;jenkins-hbase4:46313] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46313,1693152107013 2023-08-27 16:01:48,424 INFO [M:0;jenkins-hbase4:46313] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46313,1693152107013; all regions closed. 2023-08-27 16:01:48,424 DEBUG [M:0;jenkins-hbase4:46313] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-27 16:01:48,424 DEBUG [M:0;jenkins-hbase4:46313] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-27 16:01:48,424 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-27 16:01:48,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693152107202] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693152107202,5,FailOnTimeoutGroup] 2023-08-27 16:01:48,424 DEBUG [M:0;jenkins-hbase4:46313] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-27 16:01:48,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693152107203] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693152107203,5,FailOnTimeoutGroup] 2023-08-27 16:01:48,425 INFO [M:0;jenkins-hbase4:46313] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-27 16:01:48,425 INFO [M:0;jenkins-hbase4:46313] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-27 16:01:48,425 INFO [M:0;jenkins-hbase4:46313] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-27 16:01:48,426 DEBUG [M:0;jenkins-hbase4:46313] master.HMaster(1512): Stopping service threads 2023-08-27 16:01:48,426 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-27 16:01:48,426 INFO [M:0;jenkins-hbase4:46313] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-27 16:01:48,426 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-27 16:01:48,426 ERROR [M:0;jenkins-hbase4:46313] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] 2023-08-27 16:01:48,426 INFO [M:0;jenkins-hbase4:46313] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-27 16:01:48,426 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-27 16:01:48,426 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-27 16:01:48,426 DEBUG [M:0;jenkins-hbase4:46313] zookeeper.ZKUtil(398): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-27 16:01:48,427 WARN [M:0;jenkins-hbase4:46313] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-27 16:01:48,427 INFO [M:0;jenkins-hbase4:46313] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-27 16:01:48,427 INFO [M:0;jenkins-hbase4:46313] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-27 16:01:48,427 DEBUG [M:0;jenkins-hbase4:46313] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-27 16:01:48,427 INFO [M:0;jenkins-hbase4:46313] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 16:01:48,427 DEBUG [M:0;jenkins-hbase4:46313] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 16:01:48,427 DEBUG [M:0;jenkins-hbase4:46313] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-27 16:01:48,427 DEBUG [M:0;jenkins-hbase4:46313] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 16:01:48,427 INFO [M:0;jenkins-hbase4:46313] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.07 KB heapSize=29.55 KB 2023-08-27 16:01:48,437 INFO [M:0;jenkins-hbase4:46313] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.07 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/832740ed0ad849a6b37435b3315bb781 2023-08-27 16:01:48,441 DEBUG [M:0;jenkins-hbase4:46313] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/832740ed0ad849a6b37435b3315bb781 as hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/832740ed0ad849a6b37435b3315bb781 2023-08-27 16:01:48,445 INFO [M:0;jenkins-hbase4:46313] regionserver.HStore(1080): Added hdfs://localhost:34925/user/jenkins/test-data/950fd2b4-9bcf-be5f-6674-8a7310bdf8ee/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/832740ed0ad849a6b37435b3315bb781, entries=8, sequenceid=66, filesize=6.3 K 2023-08-27 16:01:48,446 INFO [M:0;jenkins-hbase4:46313] regionserver.HRegion(2948): Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 19ms, sequenceid=66, compaction requested=false 2023-08-27 16:01:48,448 INFO [M:0;jenkins-hbase4:46313] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-27 16:01:48,448 DEBUG [M:0;jenkins-hbase4:46313] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-27 16:01:48,450 INFO [M:0;jenkins-hbase4:46313] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-27 16:01:48,450 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-27 16:01:48,451 INFO [M:0;jenkins-hbase4:46313] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46313 2023-08-27 16:01:48,453 DEBUG [M:0;jenkins-hbase4:46313] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,46313,1693152107013 already deleted, retry=false 2023-08-27 16:01:48,546 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): regionserver:46435-0x1004e7a119b0001, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 16:01:48,546 INFO [RS:0;jenkins-hbase4:46435] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46435,1693152107060; zookeeper connection closed. 2023-08-27 16:01:48,547 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): regionserver:46435-0x1004e7a119b0001, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 16:01:48,547 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6c82ebd2] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6c82ebd2 2023-08-27 16:01:48,547 INFO [Listener at localhost/37897] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-27 16:01:48,647 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 16:01:48,647 INFO [M:0;jenkins-hbase4:46313] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46313,1693152107013; zookeeper connection closed. 2023-08-27 16:01:48,647 DEBUG [Listener at localhost/37897-EventThread] zookeeper.ZKWatcher(600): master:46313-0x1004e7a119b0000, quorum=127.0.0.1:65109, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-27 16:01:48,648 WARN [Listener at localhost/37897] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-27 16:01:48,652 INFO [Listener at localhost/37897] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 16:01:48,756 WARN [BP-1475386893-172.31.14.131-1693152106472 heartbeating to localhost/127.0.0.1:34925] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-27 16:01:48,756 WARN [BP-1475386893-172.31.14.131-1693152106472 heartbeating to localhost/127.0.0.1:34925] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1475386893-172.31.14.131-1693152106472 (Datanode Uuid 6e5976c8-f44a-46c8-989a-1accf7b24b38) service to localhost/127.0.0.1:34925 2023-08-27 16:01:48,756 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/cluster_6a1d2163-f9b0-ef59-34ca-1dac95c3fbf8/dfs/data/data3/current/BP-1475386893-172.31.14.131-1693152106472] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 16:01:48,757 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/cluster_6a1d2163-f9b0-ef59-34ca-1dac95c3fbf8/dfs/data/data4/current/BP-1475386893-172.31.14.131-1693152106472] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 16:01:48,757 WARN [Listener at localhost/37897] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-27 16:01:48,760 INFO [Listener at localhost/37897] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 16:01:48,863 WARN [BP-1475386893-172.31.14.131-1693152106472 heartbeating to localhost/127.0.0.1:34925] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-27 16:01:48,863 WARN [BP-1475386893-172.31.14.131-1693152106472 heartbeating to localhost/127.0.0.1:34925] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1475386893-172.31.14.131-1693152106472 (Datanode Uuid 71f7b12d-3ad6-4396-9cd6-bb4a9f905c94) service to localhost/127.0.0.1:34925 2023-08-27 16:01:48,864 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/cluster_6a1d2163-f9b0-ef59-34ca-1dac95c3fbf8/dfs/data/data1/current/BP-1475386893-172.31.14.131-1693152106472] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 16:01:48,864 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a2457502-5bf7-4e61-f505-18a316dd05d9/cluster_6a1d2163-f9b0-ef59-34ca-1dac95c3fbf8/dfs/data/data2/current/BP-1475386893-172.31.14.131-1693152106472] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-27 16:01:48,876 INFO [Listener at localhost/37897] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-27 16:01:48,987 INFO [Listener at localhost/37897] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-27 16:01:48,998 INFO [Listener at localhost/37897] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-27 16:01:49,009 INFO [Listener at localhost/37897] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=101 (was 76) - Thread LEAK? -, OpenFileDescriptor=493 (was 463) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=30 (was 24) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=12474 (was 12558)