2023-08-18 02:58:32,544 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917 2023-08-18 02:58:32,559 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.regionserver.wal.TestAsyncLogRolling timeout: 13 mins 2023-08-18 02:58:32,594 INFO [Time-limited test] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=135, ProcessCount=172, AvailableMemoryMB=3129 2023-08-18 02:58:32,602 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-18 02:58:32,602 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3, deleteOnExit=true 2023-08-18 02:58:32,603 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-18 02:58:32,603 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/test.cache.data in system properties and HBase conf 2023-08-18 02:58:32,604 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/hadoop.tmp.dir in system properties and HBase conf 2023-08-18 02:58:32,604 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/hadoop.log.dir in system properties and HBase conf 2023-08-18 02:58:32,605 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-18 02:58:32,605 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-18 02:58:32,605 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-18 02:58:32,727 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-08-18 02:58:33,117 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-18 02:58:33,121 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-18 02:58:33,121 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-18 02:58:33,122 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-18 02:58:33,122 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-18 02:58:33,122 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-18 02:58:33,122 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-18 02:58:33,123 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-18 02:58:33,123 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-18 02:58:33,124 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-18 02:58:33,124 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/nfs.dump.dir in system properties and HBase conf 2023-08-18 02:58:33,124 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/java.io.tmpdir in system properties and HBase conf 2023-08-18 02:58:33,125 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-18 02:58:33,125 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-18 02:58:33,126 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-18 02:58:33,626 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-18 02:58:33,630 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-18 02:58:33,910 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-08-18 02:58:34,076 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-08-18 02:58:34,090 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 02:58:34,125 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 02:58:34,159 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/java.io.tmpdir/Jetty_localhost_44683_hdfs____.rsrup1/webapp 2023-08-18 02:58:34,307 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44683 2023-08-18 02:58:34,326 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-18 02:58:34,327 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-18 02:58:34,834 WARN [Listener at localhost/44229] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 02:58:34,901 WARN [Listener at localhost/44229] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-18 02:58:34,919 WARN [Listener at localhost/44229] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 02:58:34,927 INFO [Listener at localhost/44229] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 02:58:34,931 INFO [Listener at localhost/44229] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/java.io.tmpdir/Jetty_localhost_42929_datanode____i6quw9/webapp 2023-08-18 02:58:35,031 INFO [Listener at localhost/44229] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42929 2023-08-18 02:58:35,356 WARN [Listener at localhost/35989] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 02:58:35,366 WARN [Listener at localhost/35989] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-18 02:58:35,370 WARN [Listener at localhost/35989] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 02:58:35,372 INFO [Listener at localhost/35989] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 02:58:35,378 INFO [Listener at localhost/35989] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/java.io.tmpdir/Jetty_localhost_32871_datanode____.w41jye/webapp 2023-08-18 02:58:35,478 INFO [Listener at localhost/35989] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:32871 2023-08-18 02:58:35,493 WARN [Listener at localhost/46011] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 02:58:35,814 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3b1fc8c1dcec31e0: Processing first storage report for DS-908f91d0-2fe4-4cee-aa46-f62342117011 from datanode ab3c0014-73dc-4192-8be5-0df7ddb087b4 2023-08-18 02:58:35,815 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3b1fc8c1dcec31e0: from storage DS-908f91d0-2fe4-4cee-aa46-f62342117011 node DatanodeRegistration(127.0.0.1:42249, datanodeUuid=ab3c0014-73dc-4192-8be5-0df7ddb087b4, infoPort=42281, infoSecurePort=0, ipcPort=46011, storageInfo=lv=-57;cid=testClusterID;nsid=1046542592;c=1692327513705), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-08-18 02:58:35,816 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xbb942932be734a1e: Processing first storage report for DS-56eb971b-aca1-45da-acb1-14a68496e5bf from datanode e2fff527-ca47-45d5-a633-f38a6996d8ca 2023-08-18 02:58:35,816 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xbb942932be734a1e: from storage DS-56eb971b-aca1-45da-acb1-14a68496e5bf node DatanodeRegistration(127.0.0.1:39037, datanodeUuid=e2fff527-ca47-45d5-a633-f38a6996d8ca, infoPort=38323, infoSecurePort=0, ipcPort=35989, storageInfo=lv=-57;cid=testClusterID;nsid=1046542592;c=1692327513705), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 02:58:35,816 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3b1fc8c1dcec31e0: Processing first storage report for DS-e65e4c6b-a5dc-43dc-bff3-167d37bacf27 from datanode ab3c0014-73dc-4192-8be5-0df7ddb087b4 2023-08-18 02:58:35,816 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3b1fc8c1dcec31e0: from storage DS-e65e4c6b-a5dc-43dc-bff3-167d37bacf27 node DatanodeRegistration(127.0.0.1:42249, datanodeUuid=ab3c0014-73dc-4192-8be5-0df7ddb087b4, infoPort=42281, infoSecurePort=0, ipcPort=46011, storageInfo=lv=-57;cid=testClusterID;nsid=1046542592;c=1692327513705), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 02:58:35,816 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xbb942932be734a1e: Processing first storage report for DS-56c76c7f-47fb-41db-ae9b-4a82ea5bbf1e from datanode e2fff527-ca47-45d5-a633-f38a6996d8ca 2023-08-18 02:58:35,816 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xbb942932be734a1e: from storage DS-56c76c7f-47fb-41db-ae9b-4a82ea5bbf1e node DatanodeRegistration(127.0.0.1:39037, datanodeUuid=e2fff527-ca47-45d5-a633-f38a6996d8ca, infoPort=38323, infoSecurePort=0, ipcPort=35989, storageInfo=lv=-57;cid=testClusterID;nsid=1046542592;c=1692327513705), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 02:58:35,885 DEBUG [Listener at localhost/46011] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917 2023-08-18 02:58:35,950 INFO [Listener at localhost/46011] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3/zookeeper_0, clientPort=55441, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-18 02:58:35,969 INFO [Listener at localhost/46011] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=55441 2023-08-18 02:58:35,976 INFO [Listener at localhost/46011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:58:35,978 INFO [Listener at localhost/46011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:58:36,657 INFO [Listener at localhost/46011] util.FSUtils(471): Created version file at hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101 with version=8 2023-08-18 02:58:36,657 INFO [Listener at localhost/46011] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/hbase-staging 2023-08-18 02:58:36,997 INFO [Listener at localhost/46011] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-08-18 02:58:37,493 INFO [Listener at localhost/46011] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-18 02:58:37,527 INFO [Listener at localhost/46011] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 02:58:37,527 INFO [Listener at localhost/46011] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-18 02:58:37,528 INFO [Listener at localhost/46011] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-18 02:58:37,528 INFO [Listener at localhost/46011] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 02:58:37,528 INFO [Listener at localhost/46011] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-18 02:58:37,673 INFO [Listener at localhost/46011] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-18 02:58:37,767 DEBUG [Listener at localhost/46011] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-08-18 02:58:37,863 INFO [Listener at localhost/46011] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41155 2023-08-18 02:58:37,873 INFO [Listener at localhost/46011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:58:37,876 INFO [Listener at localhost/46011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:58:37,901 INFO [Listener at localhost/46011] zookeeper.RecoverableZooKeeper(93): Process identifier=master:41155 connecting to ZooKeeper ensemble=127.0.0.1:55441 2023-08-18 02:58:37,944 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:411550x0, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-18 02:58:37,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:41155-0x1001d53cec50000 connected 2023-08-18 02:58:37,969 DEBUG [Listener at localhost/46011] zookeeper.ZKUtil(164): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-18 02:58:37,970 DEBUG [Listener at localhost/46011] zookeeper.ZKUtil(164): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 02:58:37,974 DEBUG [Listener at localhost/46011] zookeeper.ZKUtil(164): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-18 02:58:37,981 DEBUG [Listener at localhost/46011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41155 2023-08-18 02:58:37,982 DEBUG [Listener at localhost/46011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41155 2023-08-18 02:58:37,982 DEBUG [Listener at localhost/46011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41155 2023-08-18 02:58:37,983 DEBUG [Listener at localhost/46011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41155 2023-08-18 02:58:37,983 DEBUG [Listener at localhost/46011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41155 2023-08-18 02:58:37,989 INFO [Listener at localhost/46011] master.HMaster(444): hbase.rootdir=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101, hbase.cluster.distributed=false 2023-08-18 02:58:38,058 INFO [Listener at localhost/46011] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-18 02:58:38,058 INFO [Listener at localhost/46011] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 02:58:38,058 INFO [Listener at localhost/46011] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-18 02:58:38,059 INFO [Listener at localhost/46011] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-18 02:58:38,059 INFO [Listener at localhost/46011] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 02:58:38,059 INFO [Listener at localhost/46011] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-18 02:58:38,064 INFO [Listener at localhost/46011] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-18 02:58:38,067 INFO [Listener at localhost/46011] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33965 2023-08-18 02:58:38,069 INFO [Listener at localhost/46011] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-18 02:58:38,076 DEBUG [Listener at localhost/46011] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-18 02:58:38,077 INFO [Listener at localhost/46011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:58:38,079 INFO [Listener at localhost/46011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:58:38,081 INFO [Listener at localhost/46011] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33965 connecting to ZooKeeper ensemble=127.0.0.1:55441 2023-08-18 02:58:38,084 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): regionserver:339650x0, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-18 02:58:38,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:33965-0x1001d53cec50001 connected 2023-08-18 02:58:38,085 DEBUG [Listener at localhost/46011] zookeeper.ZKUtil(164): regionserver:33965-0x1001d53cec50001, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-18 02:58:38,086 DEBUG [Listener at localhost/46011] zookeeper.ZKUtil(164): regionserver:33965-0x1001d53cec50001, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 02:58:38,087 DEBUG [Listener at localhost/46011] zookeeper.ZKUtil(164): regionserver:33965-0x1001d53cec50001, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-18 02:58:38,088 DEBUG [Listener at localhost/46011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33965 2023-08-18 02:58:38,088 DEBUG [Listener at localhost/46011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33965 2023-08-18 02:58:38,088 DEBUG [Listener at localhost/46011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33965 2023-08-18 02:58:38,089 DEBUG [Listener at localhost/46011] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33965 2023-08-18 02:58:38,089 DEBUG [Listener at localhost/46011] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33965 2023-08-18 02:58:38,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,41155,1692327516815 2023-08-18 02:58:38,101 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-18 02:58:38,103 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,41155,1692327516815 2023-08-18 02:58:38,121 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-18 02:58:38,121 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): regionserver:33965-0x1001d53cec50001, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-18 02:58:38,121 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:58:38,122 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-18 02:58:38,124 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,41155,1692327516815 from backup master directory 2023-08-18 02:58:38,124 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-18 02:58:38,127 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,41155,1692327516815 2023-08-18 02:58:38,127 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-18 02:58:38,128 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-18 02:58:38,128 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,41155,1692327516815 2023-08-18 02:58:38,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-08-18 02:58:38,132 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-08-18 02:58:38,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/hbase.id with ID: 8f3b9239-8ac9-46f7-9d14-16c7436ffd62 2023-08-18 02:58:38,264 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:58:38,279 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:58:38,322 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x51b86828 to 127.0.0.1:55441 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-18 02:58:38,356 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@29dc4108, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-18 02:58:38,381 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-18 02:58:38,383 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-18 02:58:38,392 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-18 02:58:38,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/data/master/store-tmp 2023-08-18 02:58:38,462 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:58:38,462 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-18 02:58:38,463 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 02:58:38,463 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 02:58:38,463 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-18 02:58:38,463 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 02:58:38,463 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 02:58:38,463 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-18 02:58:38,464 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/WALs/jenkins-hbase4.apache.org,41155,1692327516815 2023-08-18 02:58:38,487 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41155%2C1692327516815, suffix=, logDir=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/WALs/jenkins-hbase4.apache.org,41155,1692327516815, archiveDir=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/oldWALs, maxLogs=10 2023-08-18 02:58:38,520 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-08-18 02:58:38,521 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-08-18 02:58:38,523 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:190) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:719) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:38,556 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:417) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:38,558 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42249,DS-908f91d0-2fe4-4cee-aa46-f62342117011,DISK] 2023-08-18 02:58:38,558 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39037,DS-56eb971b-aca1-45da-acb1-14a68496e5bf,DISK] 2023-08-18 02:58:38,565 DEBUG [RS-EventLoopGroup-3-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:38,632 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/WALs/jenkins-hbase4.apache.org,41155,1692327516815/jenkins-hbase4.apache.org%2C41155%2C1692327516815.1692327518498 2023-08-18 02:58:38,633 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42249,DS-908f91d0-2fe4-4cee-aa46-f62342117011,DISK], DatanodeInfoWithStorage[127.0.0.1:39037,DS-56eb971b-aca1-45da-acb1-14a68496e5bf,DISK]] 2023-08-18 02:58:38,633 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-18 02:58:38,634 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:58:38,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-18 02:58:38,638 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-18 02:58:38,694 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-18 02:58:38,701 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-18 02:58:38,725 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-18 02:58:38,737 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:58:38,743 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-18 02:58:38,745 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-18 02:58:38,759 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-18 02:58:38,764 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-18 02:58:38,765 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=877932, jitterRate=0.11634895205497742}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-18 02:58:38,765 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-18 02:58:38,767 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-18 02:58:38,787 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-18 02:58:38,787 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-18 02:58:38,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-18 02:58:38,792 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-08-18 02:58:38,826 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 33 msec 2023-08-18 02:58:38,826 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-18 02:58:38,852 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-18 02:58:38,857 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-18 02:58:38,882 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-18 02:58:38,885 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-18 02:58:38,888 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-18 02:58:38,893 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-18 02:58:38,897 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-18 02:58:38,900 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:58:38,902 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-18 02:58:38,903 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-18 02:58:38,915 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-18 02:58:38,919 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-18 02:58:38,919 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): regionserver:33965-0x1001d53cec50001, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-18 02:58:38,919 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:58:38,920 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,41155,1692327516815, sessionid=0x1001d53cec50000, setting cluster-up flag (Was=false) 2023-08-18 02:58:38,940 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:58:38,946 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-18 02:58:38,947 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41155,1692327516815 2023-08-18 02:58:38,957 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:58:38,963 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-18 02:58:38,964 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41155,1692327516815 2023-08-18 02:58:38,967 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/.hbase-snapshot/.tmp 2023-08-18 02:58:38,993 INFO [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(951): ClusterId : 8f3b9239-8ac9-46f7-9d14-16c7436ffd62 2023-08-18 02:58:38,998 DEBUG [RS:0;jenkins-hbase4:33965] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-18 02:58:39,004 DEBUG [RS:0;jenkins-hbase4:33965] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-18 02:58:39,004 DEBUG [RS:0;jenkins-hbase4:33965] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-18 02:58:39,007 DEBUG [RS:0;jenkins-hbase4:33965] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-18 02:58:39,008 DEBUG [RS:0;jenkins-hbase4:33965] zookeeper.ReadOnlyZKClient(139): Connect 0x0e9888cf to 127.0.0.1:55441 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-18 02:58:39,012 DEBUG [RS:0;jenkins-hbase4:33965] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5f676c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-18 02:58:39,013 DEBUG [RS:0;jenkins-hbase4:33965] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@460e3bf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-18 02:58:39,039 DEBUG [RS:0;jenkins-hbase4:33965] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:33965 2023-08-18 02:58:39,044 INFO [RS:0;jenkins-hbase4:33965] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-18 02:58:39,044 INFO [RS:0;jenkins-hbase4:33965] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-18 02:58:39,044 DEBUG [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(1022): About to register with Master. 2023-08-18 02:58:39,048 INFO [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41155,1692327516815 with isa=jenkins-hbase4.apache.org/172.31.14.131:33965, startcode=1692327518057 2023-08-18 02:58:39,066 DEBUG [RS:0;jenkins-hbase4:33965] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-18 02:58:39,096 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-18 02:58:39,106 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 02:58:39,107 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 02:58:39,107 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 02:58:39,107 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 02:58:39,107 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-18 02:58:39,107 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:58:39,107 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-18 02:58:39,108 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:58:39,109 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692327549109 2023-08-18 02:58:39,112 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-18 02:58:39,117 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-18 02:58:39,117 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-18 02:58:39,120 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45937, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-08-18 02:58:39,123 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-18 02:58:39,124 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-18 02:58:39,130 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41155] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-18 02:58:39,131 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-18 02:58:39,131 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-18 02:58:39,132 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-18 02:58:39,132 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-18 02:58:39,133 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-18 02:58:39,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-18 02:58:39,137 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-18 02:58:39,137 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-18 02:58:39,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-18 02:58:39,140 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-18 02:58:39,141 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692327519141,5,FailOnTimeoutGroup] 2023-08-18 02:58:39,142 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692327519141,5,FailOnTimeoutGroup] 2023-08-18 02:58:39,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-18 02:58:39,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-18 02:58:39,143 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-18 02:58:39,143 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-18 02:58:39,166 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-18 02:58:39,167 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-18 02:58:39,167 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101 2023-08-18 02:58:39,171 DEBUG [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(2830): Master is not running yet 2023-08-18 02:58:39,171 WARN [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-08-18 02:58:39,183 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:58:39,186 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-18 02:58:39,189 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/info 2023-08-18 02:58:39,190 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-18 02:58:39,191 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:58:39,192 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-18 02:58:39,194 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/rep_barrier 2023-08-18 02:58:39,195 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-18 02:58:39,196 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:58:39,196 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-18 02:58:39,198 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/table 2023-08-18 02:58:39,199 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-18 02:58:39,200 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:58:39,202 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740 2023-08-18 02:58:39,203 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740 2023-08-18 02:58:39,207 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-18 02:58:39,210 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-18 02:58:39,214 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-18 02:58:39,215 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=753461, jitterRate=-0.04192548990249634}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-18 02:58:39,215 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-18 02:58:39,215 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-18 02:58:39,215 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-18 02:58:39,215 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-18 02:58:39,215 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-18 02:58:39,215 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-18 02:58:39,216 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-18 02:58:39,216 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-18 02:58:39,221 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-18 02:58:39,221 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-18 02:58:39,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-18 02:58:39,241 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-18 02:58:39,244 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-18 02:58:39,272 INFO [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41155,1692327516815 with isa=jenkins-hbase4.apache.org/172.31.14.131:33965, startcode=1692327518057 2023-08-18 02:58:39,278 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41155] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33965,1692327518057 2023-08-18 02:58:39,285 DEBUG [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101 2023-08-18 02:58:39,285 DEBUG [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:44229 2023-08-18 02:58:39,285 DEBUG [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-18 02:58:39,291 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-18 02:58:39,292 DEBUG [RS:0;jenkins-hbase4:33965] zookeeper.ZKUtil(162): regionserver:33965-0x1001d53cec50001, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33965,1692327518057 2023-08-18 02:58:39,293 WARN [RS:0;jenkins-hbase4:33965] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-18 02:58:39,293 INFO [RS:0;jenkins-hbase4:33965] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-18 02:58:39,294 DEBUG [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(1948): logDir=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057 2023-08-18 02:58:39,294 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33965,1692327518057] 2023-08-18 02:58:39,302 DEBUG [RS:0;jenkins-hbase4:33965] zookeeper.ZKUtil(162): regionserver:33965-0x1001d53cec50001, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33965,1692327518057 2023-08-18 02:58:39,313 DEBUG [RS:0;jenkins-hbase4:33965] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-18 02:58:39,321 INFO [RS:0;jenkins-hbase4:33965] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-18 02:58:39,340 INFO [RS:0;jenkins-hbase4:33965] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-18 02:58:39,344 INFO [RS:0;jenkins-hbase4:33965] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-18 02:58:39,344 INFO [RS:0;jenkins-hbase4:33965] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-18 02:58:39,345 INFO [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-18 02:58:39,352 INFO [RS:0;jenkins-hbase4:33965] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-18 02:58:39,352 DEBUG [RS:0;jenkins-hbase4:33965] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:58:39,353 DEBUG [RS:0;jenkins-hbase4:33965] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:58:39,353 DEBUG [RS:0;jenkins-hbase4:33965] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:58:39,353 DEBUG [RS:0;jenkins-hbase4:33965] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:58:39,353 DEBUG [RS:0;jenkins-hbase4:33965] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:58:39,353 DEBUG [RS:0;jenkins-hbase4:33965] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-18 02:58:39,353 DEBUG [RS:0;jenkins-hbase4:33965] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:58:39,353 DEBUG [RS:0;jenkins-hbase4:33965] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:58:39,353 DEBUG [RS:0;jenkins-hbase4:33965] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:58:39,353 DEBUG [RS:0;jenkins-hbase4:33965] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:58:39,354 INFO [RS:0;jenkins-hbase4:33965] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-18 02:58:39,355 INFO [RS:0;jenkins-hbase4:33965] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-18 02:58:39,355 INFO [RS:0;jenkins-hbase4:33965] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-18 02:58:39,372 INFO [RS:0;jenkins-hbase4:33965] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-18 02:58:39,374 INFO [RS:0;jenkins-hbase4:33965] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33965,1692327518057-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-18 02:58:39,390 INFO [RS:0;jenkins-hbase4:33965] regionserver.Replication(203): jenkins-hbase4.apache.org,33965,1692327518057 started 2023-08-18 02:58:39,390 INFO [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,33965,1692327518057, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33965, sessionid=0x1001d53cec50001 2023-08-18 02:58:39,390 DEBUG [RS:0;jenkins-hbase4:33965] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-18 02:58:39,390 DEBUG [RS:0;jenkins-hbase4:33965] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33965,1692327518057 2023-08-18 02:58:39,391 DEBUG [RS:0;jenkins-hbase4:33965] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33965,1692327518057' 2023-08-18 02:58:39,391 DEBUG [RS:0;jenkins-hbase4:33965] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-18 02:58:39,391 DEBUG [RS:0;jenkins-hbase4:33965] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-18 02:58:39,392 DEBUG [RS:0;jenkins-hbase4:33965] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-18 02:58:39,392 DEBUG [RS:0;jenkins-hbase4:33965] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-18 02:58:39,392 DEBUG [RS:0;jenkins-hbase4:33965] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33965,1692327518057 2023-08-18 02:58:39,392 DEBUG [RS:0;jenkins-hbase4:33965] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33965,1692327518057' 2023-08-18 02:58:39,392 DEBUG [RS:0;jenkins-hbase4:33965] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-18 02:58:39,392 DEBUG [RS:0;jenkins-hbase4:33965] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-18 02:58:39,393 DEBUG [RS:0;jenkins-hbase4:33965] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-18 02:58:39,393 INFO [RS:0;jenkins-hbase4:33965] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-18 02:58:39,393 INFO [RS:0;jenkins-hbase4:33965] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-18 02:58:39,396 DEBUG [jenkins-hbase4:41155] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-18 02:58:39,399 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33965,1692327518057, state=OPENING 2023-08-18 02:58:39,406 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-18 02:58:39,407 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:58:39,408 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-18 02:58:39,411 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33965,1692327518057}] 2023-08-18 02:58:39,504 INFO [RS:0;jenkins-hbase4:33965] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33965%2C1692327518057, suffix=, logDir=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057, archiveDir=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/oldWALs, maxLogs=32 2023-08-18 02:58:39,524 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42249,DS-908f91d0-2fe4-4cee-aa46-f62342117011,DISK] 2023-08-18 02:58:39,524 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39037,DS-56eb971b-aca1-45da-acb1-14a68496e5bf,DISK] 2023-08-18 02:58:39,531 INFO [RS:0;jenkins-hbase4:33965] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.1692327519506 2023-08-18 02:58:39,531 DEBUG [RS:0;jenkins-hbase4:33965] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42249,DS-908f91d0-2fe4-4cee-aa46-f62342117011,DISK], DatanodeInfoWithStorage[127.0.0.1:39037,DS-56eb971b-aca1-45da-acb1-14a68496e5bf,DISK]] 2023-08-18 02:58:39,598 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33965,1692327518057 2023-08-18 02:58:39,602 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-18 02:58:39,605 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48984, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-18 02:58:39,616 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-18 02:58:39,617 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-18 02:58:39,621 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33965%2C1692327518057.meta, suffix=.meta, logDir=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057, archiveDir=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/oldWALs, maxLogs=32 2023-08-18 02:58:39,637 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42249,DS-908f91d0-2fe4-4cee-aa46-f62342117011,DISK] 2023-08-18 02:58:39,638 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39037,DS-56eb971b-aca1-45da-acb1-14a68496e5bf,DISK] 2023-08-18 02:58:39,642 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.meta.1692327519622.meta 2023-08-18 02:58:39,642 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42249,DS-908f91d0-2fe4-4cee-aa46-f62342117011,DISK], DatanodeInfoWithStorage[127.0.0.1:39037,DS-56eb971b-aca1-45da-acb1-14a68496e5bf,DISK]] 2023-08-18 02:58:39,643 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-18 02:58:39,645 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-18 02:58:39,665 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-18 02:58:39,671 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-18 02:58:39,677 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-18 02:58:39,677 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:58:39,677 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-18 02:58:39,677 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-18 02:58:39,680 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-18 02:58:39,682 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/info 2023-08-18 02:58:39,682 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/info 2023-08-18 02:58:39,683 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-18 02:58:39,684 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:58:39,684 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-18 02:58:39,686 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/rep_barrier 2023-08-18 02:58:39,686 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/rep_barrier 2023-08-18 02:58:39,686 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-18 02:58:39,687 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:58:39,687 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-18 02:58:39,689 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/table 2023-08-18 02:58:39,689 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/table 2023-08-18 02:58:39,689 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-18 02:58:39,690 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:58:39,692 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740 2023-08-18 02:58:39,695 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740 2023-08-18 02:58:39,699 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-18 02:58:39,701 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-18 02:58:39,703 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=833257, jitterRate=0.059541210532188416}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-18 02:58:39,703 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-18 02:58:39,713 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1692327519590 2023-08-18 02:58:39,731 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-18 02:58:39,732 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-18 02:58:39,732 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33965,1692327518057, state=OPEN 2023-08-18 02:58:39,735 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-18 02:58:39,735 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-18 02:58:39,740 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-18 02:58:39,740 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33965,1692327518057 in 324 msec 2023-08-18 02:58:39,745 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-18 02:58:39,745 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 511 msec 2023-08-18 02:58:39,750 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 727 msec 2023-08-18 02:58:39,750 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1692327519750, completionTime=-1 2023-08-18 02:58:39,750 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-18 02:58:39,751 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-18 02:58:39,813 DEBUG [hconnection-0x2b993733-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-18 02:58:39,816 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49000, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-18 02:58:39,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-18 02:58:39,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692327579833 2023-08-18 02:58:39,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692327639833 2023-08-18 02:58:39,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 82 msec 2023-08-18 02:58:39,858 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41155,1692327516815-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-18 02:58:39,858 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41155,1692327516815-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-18 02:58:39,858 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41155,1692327516815-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-18 02:58:39,860 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:41155, period=300000, unit=MILLISECONDS is enabled. 2023-08-18 02:58:39,861 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-18 02:58:39,867 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-18 02:58:39,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-18 02:58:39,877 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-18 02:58:39,887 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-18 02:58:39,890 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-18 02:58:39,892 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-18 02:58:39,920 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/.tmp/data/hbase/namespace/1b9d96cc9653f79a70e5ad8930936c18 2023-08-18 02:58:39,923 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/.tmp/data/hbase/namespace/1b9d96cc9653f79a70e5ad8930936c18 empty. 2023-08-18 02:58:39,924 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/.tmp/data/hbase/namespace/1b9d96cc9653f79a70e5ad8930936c18 2023-08-18 02:58:39,924 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-18 02:58:39,954 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-18 02:58:39,956 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1b9d96cc9653f79a70e5ad8930936c18, NAME => 'hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/.tmp 2023-08-18 02:58:39,973 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:58:39,973 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 1b9d96cc9653f79a70e5ad8930936c18, disabling compactions & flushes 2023-08-18 02:58:39,973 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18. 2023-08-18 02:58:39,973 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18. 2023-08-18 02:58:39,973 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18. after waiting 0 ms 2023-08-18 02:58:39,973 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18. 2023-08-18 02:58:39,973 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18. 2023-08-18 02:58:39,973 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 1b9d96cc9653f79a70e5ad8930936c18: 2023-08-18 02:58:39,978 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-18 02:58:39,994 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692327519981"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692327519981"}]},"ts":"1692327519981"} 2023-08-18 02:58:40,020 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-18 02:58:40,022 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-18 02:58:40,027 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692327520022"}]},"ts":"1692327520022"} 2023-08-18 02:58:40,031 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-18 02:58:40,040 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=1b9d96cc9653f79a70e5ad8930936c18, ASSIGN}] 2023-08-18 02:58:40,043 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=1b9d96cc9653f79a70e5ad8930936c18, ASSIGN 2023-08-18 02:58:40,045 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=1b9d96cc9653f79a70e5ad8930936c18, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33965,1692327518057; forceNewPlan=false, retain=false 2023-08-18 02:58:40,196 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=1b9d96cc9653f79a70e5ad8930936c18, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33965,1692327518057 2023-08-18 02:58:40,196 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692327520196"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692327520196"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692327520196"}]},"ts":"1692327520196"} 2023-08-18 02:58:40,202 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 1b9d96cc9653f79a70e5ad8930936c18, server=jenkins-hbase4.apache.org,33965,1692327518057}] 2023-08-18 02:58:40,364 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18. 2023-08-18 02:58:40,366 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1b9d96cc9653f79a70e5ad8930936c18, NAME => 'hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18.', STARTKEY => '', ENDKEY => ''} 2023-08-18 02:58:40,368 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 1b9d96cc9653f79a70e5ad8930936c18 2023-08-18 02:58:40,368 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:58:40,368 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1b9d96cc9653f79a70e5ad8930936c18 2023-08-18 02:58:40,368 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1b9d96cc9653f79a70e5ad8930936c18 2023-08-18 02:58:40,370 INFO [StoreOpener-1b9d96cc9653f79a70e5ad8930936c18-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1b9d96cc9653f79a70e5ad8930936c18 2023-08-18 02:58:40,373 DEBUG [StoreOpener-1b9d96cc9653f79a70e5ad8930936c18-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/namespace/1b9d96cc9653f79a70e5ad8930936c18/info 2023-08-18 02:58:40,373 DEBUG [StoreOpener-1b9d96cc9653f79a70e5ad8930936c18-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/namespace/1b9d96cc9653f79a70e5ad8930936c18/info 2023-08-18 02:58:40,373 INFO [StoreOpener-1b9d96cc9653f79a70e5ad8930936c18-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b9d96cc9653f79a70e5ad8930936c18 columnFamilyName info 2023-08-18 02:58:40,374 INFO [StoreOpener-1b9d96cc9653f79a70e5ad8930936c18-1] regionserver.HStore(310): Store=1b9d96cc9653f79a70e5ad8930936c18/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:58:40,376 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/namespace/1b9d96cc9653f79a70e5ad8930936c18 2023-08-18 02:58:40,376 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/namespace/1b9d96cc9653f79a70e5ad8930936c18 2023-08-18 02:58:40,381 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1b9d96cc9653f79a70e5ad8930936c18 2023-08-18 02:58:40,384 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/namespace/1b9d96cc9653f79a70e5ad8930936c18/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-18 02:58:40,385 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1b9d96cc9653f79a70e5ad8930936c18; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=814739, jitterRate=0.03599536418914795}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-18 02:58:40,385 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1b9d96cc9653f79a70e5ad8930936c18: 2023-08-18 02:58:40,387 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18., pid=6, masterSystemTime=1692327520356 2023-08-18 02:58:40,392 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18. 2023-08-18 02:58:40,392 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18. 2023-08-18 02:58:40,394 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=1b9d96cc9653f79a70e5ad8930936c18, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33965,1692327518057 2023-08-18 02:58:40,395 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692327520393"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692327520393"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692327520393"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692327520393"}]},"ts":"1692327520393"} 2023-08-18 02:58:40,402 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-18 02:58:40,402 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 1b9d96cc9653f79a70e5ad8930936c18, server=jenkins-hbase4.apache.org,33965,1692327518057 in 196 msec 2023-08-18 02:58:40,406 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-18 02:58:40,406 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=1b9d96cc9653f79a70e5ad8930936c18, ASSIGN in 362 msec 2023-08-18 02:58:40,407 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-18 02:58:40,408 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692327520408"}]},"ts":"1692327520408"} 2023-08-18 02:58:40,411 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-18 02:58:40,415 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-18 02:58:40,418 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 537 msec 2023-08-18 02:58:40,490 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-18 02:58:40,491 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-18 02:58:40,492 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:58:40,531 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-18 02:58:40,549 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-18 02:58:40,557 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 36 msec 2023-08-18 02:58:40,566 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-18 02:58:40,578 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-18 02:58:40,582 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 17 msec 2023-08-18 02:58:40,593 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-18 02:58:40,595 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-18 02:58:40,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.467sec 2023-08-18 02:58:40,598 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-18 02:58:40,600 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-18 02:58:40,600 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-18 02:58:40,601 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41155,1692327516815-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-18 02:58:40,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41155,1692327516815-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-18 02:58:40,613 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-18 02:58:40,700 DEBUG [Listener at localhost/46011] zookeeper.ReadOnlyZKClient(139): Connect 0x741e3453 to 127.0.0.1:55441 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-18 02:58:40,705 DEBUG [Listener at localhost/46011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@32038b2b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-18 02:58:40,739 DEBUG [hconnection-0x7910072a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-18 02:58:40,752 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49016, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-18 02:58:40,763 INFO [Listener at localhost/46011] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,41155,1692327516815 2023-08-18 02:58:40,764 INFO [Listener at localhost/46011] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:58:40,772 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-18 02:58:40,772 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:58:40,773 INFO [Listener at localhost/46011] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-18 02:58:40,787 WARN [Listener at localhost/46011] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-18 02:58:40,789 WARN [Listener at localhost/46011] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 02:58:40,791 INFO [Listener at localhost/46011] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 02:58:40,796 INFO [Listener at localhost/46011] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/java.io.tmpdir/Jetty_localhost_41415_datanode____rrjtu/webapp 2023-08-18 02:58:40,914 INFO [Listener at localhost/46011] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41415 2023-08-18 02:58:40,925 WARN [Listener at localhost/39837] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 02:58:40,949 WARN [Listener at localhost/39837] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-18 02:58:40,952 WARN [Listener at localhost/39837] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 02:58:40,954 INFO [Listener at localhost/39837] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 02:58:40,958 INFO [Listener at localhost/39837] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/java.io.tmpdir/Jetty_localhost_34811_datanode____160num/webapp 2023-08-18 02:58:41,053 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x76c3a08607cec704: Processing first storage report for DS-4eae3d22-4732-4552-ad28-c5cfec8b2908 from datanode 0c692457-357c-4267-9081-c1c02d625ebf 2023-08-18 02:58:41,053 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x76c3a08607cec704: from storage DS-4eae3d22-4732-4552-ad28-c5cfec8b2908 node DatanodeRegistration(127.0.0.1:45995, datanodeUuid=0c692457-357c-4267-9081-c1c02d625ebf, infoPort=43831, infoSecurePort=0, ipcPort=39837, storageInfo=lv=-57;cid=testClusterID;nsid=1046542592;c=1692327513705), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-18 02:58:41,053 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x76c3a08607cec704: Processing first storage report for DS-40e3fb1e-0be7-4651-b0c0-408a2fca703e from datanode 0c692457-357c-4267-9081-c1c02d625ebf 2023-08-18 02:58:41,053 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x76c3a08607cec704: from storage DS-40e3fb1e-0be7-4651-b0c0-408a2fca703e node DatanodeRegistration(127.0.0.1:45995, datanodeUuid=0c692457-357c-4267-9081-c1c02d625ebf, infoPort=43831, infoSecurePort=0, ipcPort=39837, storageInfo=lv=-57;cid=testClusterID;nsid=1046542592;c=1692327513705), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 02:58:41,081 INFO [Listener at localhost/39837] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34811 2023-08-18 02:58:41,090 WARN [Listener at localhost/36147] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 02:58:41,110 WARN [Listener at localhost/36147] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-18 02:58:41,113 WARN [Listener at localhost/36147] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 02:58:41,114 INFO [Listener at localhost/36147] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 02:58:41,124 INFO [Listener at localhost/36147] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/java.io.tmpdir/Jetty_localhost_36347_datanode____iijyfu/webapp 2023-08-18 02:58:41,206 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xff0ba575e0ca8735: Processing first storage report for DS-1b420ae4-5170-46cc-a570-2391c4d4f419 from datanode e82e5065-5e36-40c5-9a10-49abb5916be9 2023-08-18 02:58:41,206 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xff0ba575e0ca8735: from storage DS-1b420ae4-5170-46cc-a570-2391c4d4f419 node DatanodeRegistration(127.0.0.1:43873, datanodeUuid=e82e5065-5e36-40c5-9a10-49abb5916be9, infoPort=33025, infoSecurePort=0, ipcPort=36147, storageInfo=lv=-57;cid=testClusterID;nsid=1046542592;c=1692327513705), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 02:58:41,206 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xff0ba575e0ca8735: Processing first storage report for DS-327af4fd-78a2-4816-be9e-2622e8aa63ee from datanode e82e5065-5e36-40c5-9a10-49abb5916be9 2023-08-18 02:58:41,206 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xff0ba575e0ca8735: from storage DS-327af4fd-78a2-4816-be9e-2622e8aa63ee node DatanodeRegistration(127.0.0.1:43873, datanodeUuid=e82e5065-5e36-40c5-9a10-49abb5916be9, infoPort=33025, infoSecurePort=0, ipcPort=36147, storageInfo=lv=-57;cid=testClusterID;nsid=1046542592;c=1692327513705), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 02:58:41,255 INFO [Listener at localhost/36147] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36347 2023-08-18 02:58:41,266 WARN [Listener at localhost/32915] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 02:58:41,372 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xeacff61b21897153: Processing first storage report for DS-48de8558-68ea-43ae-807d-b7905c74796c from datanode 4b27bb50-c27b-4837-9ad3-d62cdfc44b2b 2023-08-18 02:58:41,372 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xeacff61b21897153: from storage DS-48de8558-68ea-43ae-807d-b7905c74796c node DatanodeRegistration(127.0.0.1:45711, datanodeUuid=4b27bb50-c27b-4837-9ad3-d62cdfc44b2b, infoPort=35917, infoSecurePort=0, ipcPort=32915, storageInfo=lv=-57;cid=testClusterID;nsid=1046542592;c=1692327513705), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 02:58:41,372 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xeacff61b21897153: Processing first storage report for DS-2cbd56b9-718d-44af-9e26-a44a3e124f35 from datanode 4b27bb50-c27b-4837-9ad3-d62cdfc44b2b 2023-08-18 02:58:41,372 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xeacff61b21897153: from storage DS-2cbd56b9-718d-44af-9e26-a44a3e124f35 node DatanodeRegistration(127.0.0.1:45711, datanodeUuid=4b27bb50-c27b-4837-9ad3-d62cdfc44b2b, infoPort=35917, infoSecurePort=0, ipcPort=32915, storageInfo=lv=-57;cid=testClusterID;nsid=1046542592;c=1692327513705), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 02:58:41,386 DEBUG [Listener at localhost/32915] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-18 02:58:41,391 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43092, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-18 02:58:41,400 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41155] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-08-18 02:58:41,400 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41155] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-08-18 02:58:41,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41155] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-18 02:58:41,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41155] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath 2023-08-18 02:58:41,408 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_PRE_OPERATION 2023-08-18 02:58:41,410 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-18 02:58:41,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41155] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRollOnDatanodeDeath" procId is: 9 2023-08-18 02:58:41,414 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/ec64483f62e1a3c18a39c4a31f1ad313 2023-08-18 02:58:41,415 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/ec64483f62e1a3c18a39c4a31f1ad313 empty. 2023-08-18 02:58:41,417 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/ec64483f62e1a3c18a39c4a31f1ad313 2023-08-18 02:58:41,417 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRollOnDatanodeDeath regions 2023-08-18 02:58:41,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41155] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-18 02:58:41,439 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/.tabledesc/.tableinfo.0000000001 2023-08-18 02:58:41,441 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(7675): creating {ENCODED => ec64483f62e1a3c18a39c4a31f1ad313, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/.tmp 2023-08-18 02:58:41,459 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:58:41,459 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1604): Closing ec64483f62e1a3c18a39c4a31f1ad313, disabling compactions & flushes 2023-08-18 02:58:41,459 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313. 2023-08-18 02:58:41,459 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313. 2023-08-18 02:58:41,459 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313. after waiting 0 ms 2023-08-18 02:58:41,459 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313. 2023-08-18 02:58:41,459 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313. 2023-08-18 02:58:41,459 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1558): Region close journal for ec64483f62e1a3c18a39c4a31f1ad313: 2023-08-18 02:58:41,463 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ADD_TO_META 2023-08-18 02:58:41,465 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692327521465"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692327521465"}]},"ts":"1692327521465"} 2023-08-18 02:58:41,468 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-18 02:58:41,470 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-18 02:58:41,470 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692327521470"}]},"ts":"1692327521470"} 2023-08-18 02:58:41,472 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLING in hbase:meta 2023-08-18 02:58:41,476 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=ec64483f62e1a3c18a39c4a31f1ad313, ASSIGN}] 2023-08-18 02:58:41,478 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=ec64483f62e1a3c18a39c4a31f1ad313, ASSIGN 2023-08-18 02:58:41,480 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=ec64483f62e1a3c18a39c4a31f1ad313, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33965,1692327518057; forceNewPlan=false, retain=false 2023-08-18 02:58:41,632 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=ec64483f62e1a3c18a39c4a31f1ad313, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33965,1692327518057 2023-08-18 02:58:41,632 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692327521632"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692327521632"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692327521632"}]},"ts":"1692327521632"} 2023-08-18 02:58:41,636 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure ec64483f62e1a3c18a39c4a31f1ad313, server=jenkins-hbase4.apache.org,33965,1692327518057}] 2023-08-18 02:58:41,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313. 2023-08-18 02:58:41,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ec64483f62e1a3c18a39c4a31f1ad313, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313.', STARTKEY => '', ENDKEY => ''} 2023-08-18 02:58:41,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRollOnDatanodeDeath ec64483f62e1a3c18a39c4a31f1ad313 2023-08-18 02:58:41,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:58:41,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ec64483f62e1a3c18a39c4a31f1ad313 2023-08-18 02:58:41,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ec64483f62e1a3c18a39c4a31f1ad313 2023-08-18 02:58:41,799 INFO [StoreOpener-ec64483f62e1a3c18a39c4a31f1ad313-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ec64483f62e1a3c18a39c4a31f1ad313 2023-08-18 02:58:41,802 DEBUG [StoreOpener-ec64483f62e1a3c18a39c4a31f1ad313-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/default/TestLogRolling-testLogRollOnDatanodeDeath/ec64483f62e1a3c18a39c4a31f1ad313/info 2023-08-18 02:58:41,802 DEBUG [StoreOpener-ec64483f62e1a3c18a39c4a31f1ad313-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/default/TestLogRolling-testLogRollOnDatanodeDeath/ec64483f62e1a3c18a39c4a31f1ad313/info 2023-08-18 02:58:41,802 INFO [StoreOpener-ec64483f62e1a3c18a39c4a31f1ad313-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec64483f62e1a3c18a39c4a31f1ad313 columnFamilyName info 2023-08-18 02:58:41,803 INFO [StoreOpener-ec64483f62e1a3c18a39c4a31f1ad313-1] regionserver.HStore(310): Store=ec64483f62e1a3c18a39c4a31f1ad313/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:58:41,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/default/TestLogRolling-testLogRollOnDatanodeDeath/ec64483f62e1a3c18a39c4a31f1ad313 2023-08-18 02:58:41,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/default/TestLogRolling-testLogRollOnDatanodeDeath/ec64483f62e1a3c18a39c4a31f1ad313 2023-08-18 02:58:41,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ec64483f62e1a3c18a39c4a31f1ad313 2023-08-18 02:58:41,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/default/TestLogRolling-testLogRollOnDatanodeDeath/ec64483f62e1a3c18a39c4a31f1ad313/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-18 02:58:41,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ec64483f62e1a3c18a39c4a31f1ad313; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=695850, jitterRate=-0.11518175899982452}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-18 02:58:41,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ec64483f62e1a3c18a39c4a31f1ad313: 2023-08-18 02:58:41,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313., pid=11, masterSystemTime=1692327521789 2023-08-18 02:58:41,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313. 2023-08-18 02:58:41,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313. 2023-08-18 02:58:41,820 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=ec64483f62e1a3c18a39c4a31f1ad313, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33965,1692327518057 2023-08-18 02:58:41,820 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692327521820"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692327521820"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692327521820"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692327521820"}]},"ts":"1692327521820"} 2023-08-18 02:58:41,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-18 02:58:41,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure ec64483f62e1a3c18a39c4a31f1ad313, server=jenkins-hbase4.apache.org,33965,1692327518057 in 187 msec 2023-08-18 02:58:41,830 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-18 02:58:41,831 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=ec64483f62e1a3c18a39c4a31f1ad313, ASSIGN in 350 msec 2023-08-18 02:58:41,832 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-18 02:58:41,832 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692327521832"}]},"ts":"1692327521832"} 2023-08-18 02:58:41,835 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLED in hbase:meta 2023-08-18 02:58:41,839 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_POST_OPERATION 2023-08-18 02:58:41,841 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath in 435 msec 2023-08-18 02:58:45,190 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-18 02:58:45,319 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-18 02:58:45,321 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-18 02:58:45,322 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRollOnDatanodeDeath' 2023-08-18 02:58:46,992 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-18 02:58:46,993 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-08-18 02:58:51,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41155] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-18 02:58:51,434 INFO [Listener at localhost/32915] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRollOnDatanodeDeath, procId: 9 completed 2023-08-18 02:58:51,434 DEBUG [Listener at localhost/32915] hbase.HBaseTestingUtility(3430): Waiting until all regions of table TestLogRolling-testLogRollOnDatanodeDeath get assigned. Timeout = 60000ms 2023-08-18 02:58:51,436 INFO [Listener at localhost/32915] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-08-18 02:58:51,440 INFO [Listener at localhost/32915] hbase.HBaseTestingUtility(3484): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned to meta. Checking AM states. 2023-08-18 02:58:51,441 INFO [Listener at localhost/32915] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-08-18 02:58:51,441 INFO [Listener at localhost/32915] hbase.HBaseTestingUtility(3504): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned. 2023-08-18 02:58:51,452 DEBUG [Listener at localhost/32915] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRollOnDatanodeDeath 2023-08-18 02:58:51,454 DEBUG [Listener at localhost/32915] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313. 2023-08-18 02:58:51,454 WARN [Listener at localhost/32915] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-18 02:58:51,457 INFO [Listener at localhost/32915] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 02:58:51,463 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1989492226_17 at /127.0.0.1:57664 [Receiving block BP-1884514230-172.31.14.131-1692327513705:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:39037:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:57664 dst: /127.0.0.1:39037 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:51,463 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_374674475_17 at /127.0.0.1:57676 [Receiving block BP-1884514230-172.31.14.131-1692327513705:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:39037:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:57676 dst: /127.0.0.1:39037 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:51,463 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_374674475_17 at /127.0.0.1:57686 [Receiving block BP-1884514230-172.31.14.131-1692327513705:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:39037:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:57686 dst: /127.0.0.1:39037 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:51,560 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_374674475_17 at /127.0.0.1:33440 [Receiving block BP-1884514230-172.31.14.131-1692327513705:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:42249:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:33440 dst: /127.0.0.1:42249 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:51,561 WARN [BP-1884514230-172.31.14.131-1692327513705 heartbeating to localhost/127.0.0.1:44229] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-18 02:58:51,561 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_374674475_17 at /127.0.0.1:33424 [Receiving block BP-1884514230-172.31.14.131-1692327513705:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:42249:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:33424 dst: /127.0.0.1:42249 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:51,560 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1989492226_17 at /127.0.0.1:33394 [Receiving block BP-1884514230-172.31.14.131-1692327513705:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:42249:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:33394 dst: /127.0.0.1:42249 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:51,561 WARN [BP-1884514230-172.31.14.131-1692327513705 heartbeating to localhost/127.0.0.1:44229] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1884514230-172.31.14.131-1692327513705 (Datanode Uuid ab3c0014-73dc-4192-8be5-0df7ddb087b4) service to localhost/127.0.0.1:44229 2023-08-18 02:58:51,564 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3/dfs/data/data3/current/BP-1884514230-172.31.14.131-1692327513705] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 02:58:51,564 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3/dfs/data/data4/current/BP-1884514230-172.31.14.131-1692327513705] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 02:58:51,572 WARN [Listener at localhost/32915] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-18 02:58:51,574 WARN [Listener at localhost/32915] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 02:58:51,575 INFO [Listener at localhost/32915] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 02:58:51,580 INFO [Listener at localhost/32915] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/java.io.tmpdir/Jetty_localhost_34523_datanode____.knwndw/webapp 2023-08-18 02:58:51,675 INFO [Listener at localhost/32915] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34523 2023-08-18 02:58:51,684 WARN [Listener at localhost/37847] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 02:58:51,688 WARN [AsyncFSWAL-0-hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101-prefix:jenkins-hbase4.apache.org,33965,1692327518057] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:51,690 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C33965%2C1692327518057:(num 1692327519506) roll requested 2023-08-18 02:58:51,708 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43873,DS-1b420ae4-5170-46cc-a570-2391c4d4f419,DISK] 2023-08-18 02:58:51,708 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45711,DS-48de8558-68ea-43ae-807d-b7905c74796c,DISK] 2023-08-18 02:58:51,713 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.1692327519506 with entries=6, filesize=3.19 KB; new WAL /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.1692327531690 2023-08-18 02:58:51,714 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45711,DS-48de8558-68ea-43ae-807d-b7905c74796c,DISK], DatanodeInfoWithStorage[127.0.0.1:43873,DS-1b420ae4-5170-46cc-a570-2391c4d4f419,DISK]] 2023-08-18 02:58:51,714 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.1692327519506 is not closed yet, will try archiving it next time 2023-08-18 02:58:51,715 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-18 02:58:51,716 INFO [Listener at localhost/37847] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-18 02:58:51,717 DEBUG [Listener at localhost/37847] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x741e3453 to 127.0.0.1:55441 2023-08-18 02:58:51,717 DEBUG [Listener at localhost/37847] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 02:58:51,717 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:51,718 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:51,718 DEBUG [Listener at localhost/37847] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-18 02:58:51,718 DEBUG [Listener at localhost/37847] util.JVMClusterUtil(257): Found active master hash=2042409877, stopped=false 2023-08-18 02:58:51,718 INFO [Listener at localhost/37847] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,41155,1692327516815 2023-08-18 02:58:51,721 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.1692327519506 2023-08-18 02:58:51,721 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): regionserver:33965-0x1001d53cec50001, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-18 02:58:51,721 INFO [Listener at localhost/37847] procedure2.ProcedureExecutor(629): Stopping 2023-08-18 02:58:51,721 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-18 02:58:51,721 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:58:51,721 DEBUG [Listener at localhost/37847] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x51b86828 to 127.0.0.1:55441 2023-08-18 02:58:51,722 DEBUG [Listener at localhost/37847] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 02:58:51,722 INFO [Listener at localhost/37847] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33965,1692327518057' ***** 2023-08-18 02:58:51,722 INFO [Listener at localhost/37847] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-18 02:58:51,722 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33965-0x1001d53cec50001, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 02:58:51,722 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 02:58:51,725 INFO [RS:0;jenkins-hbase4:33965] regionserver.HeapMemoryManager(220): Stopping 2023-08-18 02:58:51,727 INFO [RS:0;jenkins-hbase4:33965] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-18 02:58:51,727 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-18 02:58:51,727 INFO [RS:0;jenkins-hbase4:33965] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-18 02:58:51,727 INFO [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(3305): Received CLOSE for 1b9d96cc9653f79a70e5ad8930936c18 2023-08-18 02:58:51,728 INFO [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(3305): Received CLOSE for ec64483f62e1a3c18a39c4a31f1ad313 2023-08-18 02:58:51,729 INFO [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33965,1692327518057 2023-08-18 02:58:51,729 DEBUG [RS:0;jenkins-hbase4:33965] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0e9888cf to 127.0.0.1:55441 2023-08-18 02:58:51,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1b9d96cc9653f79a70e5ad8930936c18, disabling compactions & flushes 2023-08-18 02:58:51,729 DEBUG [RS:0;jenkins-hbase4:33965] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 02:58:51,729 INFO [RS:0;jenkins-hbase4:33965] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-18 02:58:51,729 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18. 2023-08-18 02:58:51,729 INFO [RS:0;jenkins-hbase4:33965] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-18 02:58:51,730 WARN [IPC Server handler 3 on default port 44229] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.1692327519506 has not been closed. Lease recovery is in progress. RecoveryId = 1015 for block blk_1073741832_1008 2023-08-18 02:58:51,730 INFO [RS:0;jenkins-hbase4:33965] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-18 02:58:51,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18. 2023-08-18 02:58:51,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18. after waiting 0 ms 2023-08-18 02:58:51,730 INFO [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-18 02:58:51,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18. 2023-08-18 02:58:51,730 INFO [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-08-18 02:58:51,731 DEBUG [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(1478): Online Regions={1b9d96cc9653f79a70e5ad8930936c18=hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18., 1588230740=hbase:meta,,1.1588230740, ec64483f62e1a3c18a39c4a31f1ad313=TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313.} 2023-08-18 02:58:51,731 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-18 02:58:51,731 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-18 02:58:51,731 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-18 02:58:51,731 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1b9d96cc9653f79a70e5ad8930936c18 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-18 02:58:51,731 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-18 02:58:51,732 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-18 02:58:51,732 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.92 KB heapSize=5.45 KB 2023-08-18 02:58:51,733 DEBUG [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(1504): Waiting on 1588230740, 1b9d96cc9653f79a70e5ad8930936c18, ec64483f62e1a3c18a39c4a31f1ad313 2023-08-18 02:58:51,733 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.1692327519506 after 12ms 2023-08-18 02:58:51,741 WARN [AsyncFSWAL-0-hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101-prefix:jenkins-hbase4.apache.org,33965,1692327518057.meta] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:51,742 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C33965%2C1692327518057.meta:.meta(num 1692327519622) roll requested 2023-08-18 02:58:51,762 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43873,DS-1b420ae4-5170-46cc-a570-2391c4d4f419,DISK] 2023-08-18 02:58:51,762 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45711,DS-48de8558-68ea-43ae-807d-b7905c74796c,DISK] 2023-08-18 02:58:51,770 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.meta.1692327519622.meta with entries=12, filesize=3.87 KB; new WAL /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.meta.1692327531742.meta 2023-08-18 02:58:51,773 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45711,DS-48de8558-68ea-43ae-807d-b7905c74796c,DISK], DatanodeInfoWithStorage[127.0.0.1:43873,DS-1b420ae4-5170-46cc-a570-2391c4d4f419,DISK]] 2023-08-18 02:58:51,773 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.meta.1692327519622.meta is not closed yet, will try archiving it next time 2023-08-18 02:58:51,773 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:51,774 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:51,777 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.meta.1692327519622.meta 2023-08-18 02:58:51,780 WARN [IPC Server handler 1 on default port 44229] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.meta.1692327519622.meta has not been closed. Lease recovery is in progress. RecoveryId = 1017 for block blk_1073741833_1009 2023-08-18 02:58:51,780 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.meta.1692327519622.meta after 1ms 2023-08-18 02:58:51,866 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/namespace/1b9d96cc9653f79a70e5ad8930936c18/.tmp/info/1b35995e13b2485799305296e737c1be 2023-08-18 02:58:51,872 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.69 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/.tmp/info/7bc9adef10f14fd0a7dce5df9458aa36 2023-08-18 02:58:51,883 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb351d07eaa0f54ba: Processing first storage report for DS-908f91d0-2fe4-4cee-aa46-f62342117011 from datanode ab3c0014-73dc-4192-8be5-0df7ddb087b4 2023-08-18 02:58:51,884 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb351d07eaa0f54ba: from storage DS-908f91d0-2fe4-4cee-aa46-f62342117011 node DatanodeRegistration(127.0.0.1:43043, datanodeUuid=ab3c0014-73dc-4192-8be5-0df7ddb087b4, infoPort=38205, infoSecurePort=0, ipcPort=37847, storageInfo=lv=-57;cid=testClusterID;nsid=1046542592;c=1692327513705), blocks: 6, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-18 02:58:51,884 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb351d07eaa0f54ba: Processing first storage report for DS-e65e4c6b-a5dc-43dc-bff3-167d37bacf27 from datanode ab3c0014-73dc-4192-8be5-0df7ddb087b4 2023-08-18 02:58:51,884 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb351d07eaa0f54ba: from storage DS-e65e4c6b-a5dc-43dc-bff3-167d37bacf27 node DatanodeRegistration(127.0.0.1:43043, datanodeUuid=ab3c0014-73dc-4192-8be5-0df7ddb087b4, infoPort=38205, infoSecurePort=0, ipcPort=37847, storageInfo=lv=-57;cid=testClusterID;nsid=1046542592;c=1692327513705), blocks: 7, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-18 02:58:51,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/namespace/1b9d96cc9653f79a70e5ad8930936c18/.tmp/info/1b35995e13b2485799305296e737c1be as hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/namespace/1b9d96cc9653f79a70e5ad8930936c18/info/1b35995e13b2485799305296e737c1be 2023-08-18 02:58:51,933 DEBUG [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(1504): Waiting on 1588230740, 1b9d96cc9653f79a70e5ad8930936c18, ec64483f62e1a3c18a39c4a31f1ad313 2023-08-18 02:58:51,936 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/namespace/1b9d96cc9653f79a70e5ad8930936c18/info/1b35995e13b2485799305296e737c1be, entries=2, sequenceid=6, filesize=4.8 K 2023-08-18 02:58:51,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 1b9d96cc9653f79a70e5ad8930936c18 in 210ms, sequenceid=6, compaction requested=false 2023-08-18 02:58:51,945 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/.tmp/table/a7c722ba98af464ba4e2857036b05ea8 2023-08-18 02:58:51,956 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/.tmp/info/7bc9adef10f14fd0a7dce5df9458aa36 as hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/info/7bc9adef10f14fd0a7dce5df9458aa36 2023-08-18 02:58:51,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/namespace/1b9d96cc9653f79a70e5ad8930936c18/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-18 02:58:51,959 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18. 2023-08-18 02:58:51,959 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1b9d96cc9653f79a70e5ad8930936c18: 2023-08-18 02:58:51,959 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1692327519876.1b9d96cc9653f79a70e5ad8930936c18. 2023-08-18 02:58:51,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ec64483f62e1a3c18a39c4a31f1ad313, disabling compactions & flushes 2023-08-18 02:58:51,960 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313. 2023-08-18 02:58:51,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313. 2023-08-18 02:58:51,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313. after waiting 0 ms 2023-08-18 02:58:51,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313. 2023-08-18 02:58:51,960 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ec64483f62e1a3c18a39c4a31f1ad313 1/1 column families, dataSize=2.05 KB heapSize=2.45 KB 2023-08-18 02:58:51,967 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/info/7bc9adef10f14fd0a7dce5df9458aa36, entries=20, sequenceid=14, filesize=7.4 K 2023-08-18 02:58:51,970 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/.tmp/table/a7c722ba98af464ba4e2857036b05ea8 as hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/table/a7c722ba98af464ba4e2857036b05ea8 2023-08-18 02:58:51,983 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/table/a7c722ba98af464ba4e2857036b05ea8, entries=4, sequenceid=14, filesize=4.8 K 2023-08-18 02:58:51,985 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.92 KB/2994, heapSize ~5.16 KB/5288, currentSize=0 B/0 for 1588230740 in 253ms, sequenceid=14, compaction requested=false 2023-08-18 02:58:51,985 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.05 KB at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/default/TestLogRolling-testLogRollOnDatanodeDeath/ec64483f62e1a3c18a39c4a31f1ad313/.tmp/info/7c1b257526434ed5835e14b466f4deb8 2023-08-18 02:58:51,998 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-08-18 02:58:51,999 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-18 02:58:52,000 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-18 02:58:52,000 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-18 02:58:52,000 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-18 02:58:52,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/default/TestLogRolling-testLogRollOnDatanodeDeath/ec64483f62e1a3c18a39c4a31f1ad313/.tmp/info/7c1b257526434ed5835e14b466f4deb8 as hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/default/TestLogRolling-testLogRollOnDatanodeDeath/ec64483f62e1a3c18a39c4a31f1ad313/info/7c1b257526434ed5835e14b466f4deb8 2023-08-18 02:58:52,010 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/default/TestLogRolling-testLogRollOnDatanodeDeath/ec64483f62e1a3c18a39c4a31f1ad313/info/7c1b257526434ed5835e14b466f4deb8, entries=2, sequenceid=6, filesize=6.8 K 2023-08-18 02:58:52,012 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.05 KB/2102, heapSize ~2.44 KB/2496, currentSize=0 B/0 for ec64483f62e1a3c18a39c4a31f1ad313 in 52ms, sequenceid=6, compaction requested=false 2023-08-18 02:58:52,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/data/default/TestLogRolling-testLogRollOnDatanodeDeath/ec64483f62e1a3c18a39c4a31f1ad313/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-18 02:58:52,021 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313. 2023-08-18 02:58:52,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ec64483f62e1a3c18a39c4a31f1ad313: 2023-08-18 02:58:52,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1692327521399.ec64483f62e1a3c18a39c4a31f1ad313. 2023-08-18 02:58:52,133 INFO [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33965,1692327518057; all regions closed. 2023-08-18 02:58:52,355 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-18 02:58:52,355 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-18 02:58:53,357 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-18 02:58:55,734 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.1692327519506 after 4013ms 2023-08-18 02:58:55,735 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.1692327519506 to hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/oldWALs/jenkins-hbase4.apache.org%2C33965%2C1692327518057.1692327519506 2023-08-18 02:58:55,782 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.meta.1692327519622.meta after 4003ms 2023-08-18 02:58:55,783 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/WALs/jenkins-hbase4.apache.org,33965,1692327518057/jenkins-hbase4.apache.org%2C33965%2C1692327518057.meta.1692327519622.meta to hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/oldWALs/jenkins-hbase4.apache.org%2C33965%2C1692327518057.meta.1692327519622.meta 2023-08-18 02:58:55,789 DEBUG [RS:0;jenkins-hbase4:33965] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/oldWALs 2023-08-18 02:58:55,789 INFO [RS:0;jenkins-hbase4:33965] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33965%2C1692327518057.meta:.meta(num 1692327531742) 2023-08-18 02:58:55,799 DEBUG [RS:0;jenkins-hbase4:33965] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/oldWALs 2023-08-18 02:58:55,799 INFO [RS:0;jenkins-hbase4:33965] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33965%2C1692327518057:(num 1692327531690) 2023-08-18 02:58:55,799 DEBUG [RS:0;jenkins-hbase4:33965] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 02:58:55,799 INFO [RS:0;jenkins-hbase4:33965] regionserver.LeaseManager(133): Closed leases 2023-08-18 02:58:55,800 INFO [RS:0;jenkins-hbase4:33965] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-18 02:58:55,800 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-18 02:58:55,800 INFO [RS:0;jenkins-hbase4:33965] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33965 2023-08-18 02:58:55,808 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): regionserver:33965-0x1001d53cec50001, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33965,1692327518057 2023-08-18 02:58:55,808 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-18 02:58:55,808 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): regionserver:33965-0x1001d53cec50001, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-18 02:58:55,811 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33965,1692327518057] 2023-08-18 02:58:55,811 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33965,1692327518057; numProcessing=1 2023-08-18 02:58:55,813 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,33965,1692327518057 already deleted, retry=false 2023-08-18 02:58:55,813 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33965,1692327518057 expired; onlineServers=0 2023-08-18 02:58:55,813 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41155,1692327516815' ***** 2023-08-18 02:58:55,813 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-18 02:58:55,813 DEBUG [M:0;jenkins-hbase4:41155] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3881b9ae, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-18 02:58:55,813 INFO [M:0;jenkins-hbase4:41155] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41155,1692327516815 2023-08-18 02:58:55,814 INFO [M:0;jenkins-hbase4:41155] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41155,1692327516815; all regions closed. 2023-08-18 02:58:55,814 DEBUG [M:0;jenkins-hbase4:41155] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 02:58:55,814 DEBUG [M:0;jenkins-hbase4:41155] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-18 02:58:55,814 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-18 02:58:55,814 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692327519141] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692327519141,5,FailOnTimeoutGroup] 2023-08-18 02:58:55,814 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692327519141] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692327519141,5,FailOnTimeoutGroup] 2023-08-18 02:58:55,814 DEBUG [M:0;jenkins-hbase4:41155] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-18 02:58:55,816 INFO [M:0;jenkins-hbase4:41155] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-18 02:58:55,816 INFO [M:0;jenkins-hbase4:41155] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-18 02:58:55,816 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-18 02:58:55,816 INFO [M:0;jenkins-hbase4:41155] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-18 02:58:55,816 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:58:55,817 DEBUG [M:0;jenkins-hbase4:41155] master.HMaster(1512): Stopping service threads 2023-08-18 02:58:55,817 INFO [M:0;jenkins-hbase4:41155] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-18 02:58:55,817 ERROR [M:0;jenkins-hbase4:41155] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] 2023-08-18 02:58:55,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-18 02:58:55,818 INFO [M:0;jenkins-hbase4:41155] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-18 02:58:55,818 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-18 02:58:55,818 DEBUG [M:0;jenkins-hbase4:41155] zookeeper.ZKUtil(398): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-18 02:58:55,819 WARN [M:0;jenkins-hbase4:41155] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-18 02:58:55,819 INFO [M:0;jenkins-hbase4:41155] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-18 02:58:55,819 INFO [M:0;jenkins-hbase4:41155] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-18 02:58:55,819 DEBUG [M:0;jenkins-hbase4:41155] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-18 02:58:55,819 INFO [M:0;jenkins-hbase4:41155] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 02:58:55,819 DEBUG [M:0;jenkins-hbase4:41155] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 02:58:55,819 DEBUG [M:0;jenkins-hbase4:41155] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-18 02:58:55,820 DEBUG [M:0;jenkins-hbase4:41155] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 02:58:55,820 INFO [M:0;jenkins-hbase4:41155] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.07 KB heapSize=45.73 KB 2023-08-18 02:58:55,820 WARN [AsyncFSWAL-0-hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData-prefix:jenkins-hbase4.apache.org,41155,1692327516815] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:55,821 DEBUG [master:store-WAL-Roller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C41155%2C1692327516815:(num 1692327518498) roll requested 2023-08-18 02:58:55,839 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45995,DS-4eae3d22-4732-4552-ad28-c5cfec8b2908,DISK] 2023-08-18 02:58:55,839 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43873,DS-1b420ae4-5170-46cc-a570-2391c4d4f419,DISK] 2023-08-18 02:58:55,843 INFO [master:store-WAL-Roller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/WALs/jenkins-hbase4.apache.org,41155,1692327516815/jenkins-hbase4.apache.org%2C41155%2C1692327516815.1692327518498 with entries=89, filesize=43.92 KB; new WAL /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/WALs/jenkins-hbase4.apache.org,41155,1692327516815/jenkins-hbase4.apache.org%2C41155%2C1692327516815.1692327535821 2023-08-18 02:58:55,843 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45995,DS-4eae3d22-4732-4552-ad28-c5cfec8b2908,DISK], DatanodeInfoWithStorage[127.0.0.1:43873,DS-1b420ae4-5170-46cc-a570-2391c4d4f419,DISK]] 2023-08-18 02:58:55,843 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(716): hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/WALs/jenkins-hbase4.apache.org,41155,1692327516815/jenkins-hbase4.apache.org%2C41155%2C1692327516815.1692327518498 is not closed yet, will try archiving it next time 2023-08-18 02:58:55,843 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:55,844 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-18 02:58:55,844 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/WALs/jenkins-hbase4.apache.org,41155,1692327516815/jenkins-hbase4.apache.org%2C41155%2C1692327516815.1692327518498 2023-08-18 02:58:55,846 WARN [IPC Server handler 0 on default port 44229] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/WALs/jenkins-hbase4.apache.org,41155,1692327516815/jenkins-hbase4.apache.org%2C41155%2C1692327516815.1692327518498 has not been closed. Lease recovery is in progress. RecoveryId = 1023 for block blk_1073741829_1005 2023-08-18 02:58:55,846 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/WALs/jenkins-hbase4.apache.org,41155,1692327516815/jenkins-hbase4.apache.org%2C41155%2C1692327516815.1692327518498 after 2ms 2023-08-18 02:58:55,860 INFO [M:0;jenkins-hbase4:41155] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.07 KB at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5db0f476220f409eb4a85cbb733d9c45 2023-08-18 02:58:55,868 DEBUG [M:0;jenkins-hbase4:41155] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5db0f476220f409eb4a85cbb733d9c45 as hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5db0f476220f409eb4a85cbb733d9c45 2023-08-18 02:58:55,875 INFO [M:0;jenkins-hbase4:41155] regionserver.HStore(1080): Added hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5db0f476220f409eb4a85cbb733d9c45, entries=11, sequenceid=91, filesize=7.0 K 2023-08-18 02:58:55,876 INFO [M:0;jenkins-hbase4:41155] regionserver.HRegion(2948): Finished flush of dataSize ~38.07 KB/38985, heapSize ~45.72 KB/46816, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 56ms, sequenceid=91, compaction requested=false 2023-08-18 02:58:55,878 INFO [M:0;jenkins-hbase4:41155] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 02:58:55,878 DEBUG [M:0;jenkins-hbase4:41155] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-18 02:58:55,911 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): regionserver:33965-0x1001d53cec50001, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 02:58:55,911 INFO [RS:0;jenkins-hbase4:33965] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33965,1692327518057; zookeeper connection closed. 2023-08-18 02:58:55,912 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): regionserver:33965-0x1001d53cec50001, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 02:58:55,912 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@578f92fa] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@578f92fa 2023-08-18 02:58:55,913 INFO [Listener at localhost/37847] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-18 02:58:57,505 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-18 02:58:57,542 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-18 02:58:59,847 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/WALs/jenkins-hbase4.apache.org,41155,1692327516815/jenkins-hbase4.apache.org%2C41155%2C1692327516815.1692327518498 after 4003ms 2023-08-18 02:58:59,848 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/WALs/jenkins-hbase4.apache.org,41155,1692327516815/jenkins-hbase4.apache.org%2C41155%2C1692327516815.1692327518498 to hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/oldWALs/jenkins-hbase4.apache.org%2C41155%2C1692327516815.1692327518498 2023-08-18 02:58:59,854 INFO [WAL-Archive-0] region.MasterRegionUtils(50): Moved hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/MasterData/oldWALs/jenkins-hbase4.apache.org%2C41155%2C1692327516815.1692327518498 to hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/oldWALs/jenkins-hbase4.apache.org%2C41155%2C1692327516815.1692327518498$masterlocalwal$ 2023-08-18 02:58:59,854 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-18 02:58:59,854 INFO [M:0;jenkins-hbase4:41155] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-18 02:58:59,855 INFO [M:0;jenkins-hbase4:41155] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41155 2023-08-18 02:58:59,858 DEBUG [M:0;jenkins-hbase4:41155] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,41155,1692327516815 already deleted, retry=false 2023-08-18 02:58:59,962 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 02:58:59,962 INFO [M:0;jenkins-hbase4:41155] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41155,1692327516815; zookeeper connection closed. 2023-08-18 02:58:59,962 DEBUG [Listener at localhost/46011-EventThread] zookeeper.ZKWatcher(600): master:41155-0x1001d53cec50000, quorum=127.0.0.1:55441, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 02:58:59,965 WARN [Listener at localhost/37847] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-18 02:58:59,969 INFO [Listener at localhost/37847] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 02:59:00,072 WARN [BP-1884514230-172.31.14.131-1692327513705 heartbeating to localhost/127.0.0.1:44229] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-18 02:59:00,072 WARN [BP-1884514230-172.31.14.131-1692327513705 heartbeating to localhost/127.0.0.1:44229] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1884514230-172.31.14.131-1692327513705 (Datanode Uuid ab3c0014-73dc-4192-8be5-0df7ddb087b4) service to localhost/127.0.0.1:44229 2023-08-18 02:59:00,073 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3/dfs/data/data3/current/BP-1884514230-172.31.14.131-1692327513705] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 02:59:00,073 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3/dfs/data/data4/current/BP-1884514230-172.31.14.131-1692327513705] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 02:59:00,076 WARN [Listener at localhost/37847] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-18 02:59:00,080 INFO [Listener at localhost/37847] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 02:59:00,183 WARN [BP-1884514230-172.31.14.131-1692327513705 heartbeating to localhost/127.0.0.1:44229] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-18 02:59:00,183 WARN [BP-1884514230-172.31.14.131-1692327513705 heartbeating to localhost/127.0.0.1:44229] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1884514230-172.31.14.131-1692327513705 (Datanode Uuid 4b27bb50-c27b-4837-9ad3-d62cdfc44b2b) service to localhost/127.0.0.1:44229 2023-08-18 02:59:00,184 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3/dfs/data/data9/current/BP-1884514230-172.31.14.131-1692327513705] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 02:59:00,184 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3/dfs/data/data10/current/BP-1884514230-172.31.14.131-1692327513705] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 02:59:00,185 WARN [Listener at localhost/37847] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-18 02:59:00,189 INFO [Listener at localhost/37847] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 02:59:00,293 WARN [BP-1884514230-172.31.14.131-1692327513705 heartbeating to localhost/127.0.0.1:44229] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-18 02:59:00,294 WARN [BP-1884514230-172.31.14.131-1692327513705 heartbeating to localhost/127.0.0.1:44229] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1884514230-172.31.14.131-1692327513705 (Datanode Uuid e82e5065-5e36-40c5-9a10-49abb5916be9) service to localhost/127.0.0.1:44229 2023-08-18 02:59:00,294 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3/dfs/data/data7/current/BP-1884514230-172.31.14.131-1692327513705] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 02:59:00,295 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3/dfs/data/data8/current/BP-1884514230-172.31.14.131-1692327513705] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 02:59:00,296 WARN [Listener at localhost/37847] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-18 02:59:00,299 INFO [Listener at localhost/37847] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 02:59:00,403 WARN [BP-1884514230-172.31.14.131-1692327513705 heartbeating to localhost/127.0.0.1:44229] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-18 02:59:00,403 WARN [BP-1884514230-172.31.14.131-1692327513705 heartbeating to localhost/127.0.0.1:44229] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1884514230-172.31.14.131-1692327513705 (Datanode Uuid 0c692457-357c-4267-9081-c1c02d625ebf) service to localhost/127.0.0.1:44229 2023-08-18 02:59:00,403 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3/dfs/data/data5/current/BP-1884514230-172.31.14.131-1692327513705] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 02:59:00,404 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3/dfs/data/data6/current/BP-1884514230-172.31.14.131-1692327513705] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 02:59:00,405 WARN [Listener at localhost/37847] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-18 02:59:00,407 INFO [Listener at localhost/37847] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 02:59:00,508 WARN [BP-1884514230-172.31.14.131-1692327513705 heartbeating to localhost/127.0.0.1:44229] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-18 02:59:00,508 WARN [BP-1884514230-172.31.14.131-1692327513705 heartbeating to localhost/127.0.0.1:44229] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1884514230-172.31.14.131-1692327513705 (Datanode Uuid e2fff527-ca47-45d5-a633-f38a6996d8ca) service to localhost/127.0.0.1:44229 2023-08-18 02:59:00,509 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3/dfs/data/data1/current/BP-1884514230-172.31.14.131-1692327513705] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 02:59:00,509 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/cluster_7d3e519e-3a73-d8d1-fc92-812044f211c3/dfs/data/data2/current/BP-1884514230-172.31.14.131-1692327513705] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 02:59:00,540 INFO [Listener at localhost/37847] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 02:59:00,656 INFO [Listener at localhost/37847] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-18 02:59:00,703 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-18 02:59:00,717 INFO [Listener at localhost/37847] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=78 (was 10) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: ForkJoinPool-2-worker-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:44229 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:44229 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (531659913) connection to localhost/127.0.0.1:44229 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/37847 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (531659913) connection to localhost/127.0.0.1:44229 from jenkins.hfs.0 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (531659913) connection to localhost/127.0.0.1:44229 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@1430dce1 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=467 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=119 (was 135), ProcessCount=172 (was 172), AvailableMemoryMB=2337 (was 3129) 2023-08-18 02:59:00,727 INFO [Listener at localhost/37847] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=78, OpenFileDescriptor=467, MaxFileDescriptor=60000, SystemLoadAverage=119, ProcessCount=172, AvailableMemoryMB=2337 2023-08-18 02:59:00,728 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-18 02:59:00,728 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/hadoop.log.dir so I do NOT create it in target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa 2023-08-18 02:59:00,728 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ec2a32b-1c0b-31c4-dbe2-d8cc7aeb3917/hadoop.tmp.dir so I do NOT create it in target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa 2023-08-18 02:59:00,728 INFO [Listener at localhost/37847] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/cluster_8224e203-f1af-f19e-05b6-812bd3cec471, deleteOnExit=true 2023-08-18 02:59:00,728 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-18 02:59:00,728 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/test.cache.data in system properties and HBase conf 2023-08-18 02:59:00,729 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/hadoop.tmp.dir in system properties and HBase conf 2023-08-18 02:59:00,729 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/hadoop.log.dir in system properties and HBase conf 2023-08-18 02:59:00,729 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-18 02:59:00,729 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-18 02:59:00,729 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-18 02:59:00,729 DEBUG [Listener at localhost/37847] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-18 02:59:00,730 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-18 02:59:00,730 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-18 02:59:00,730 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-18 02:59:00,730 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-18 02:59:00,730 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-18 02:59:00,730 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-18 02:59:00,730 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-18 02:59:00,730 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-18 02:59:00,730 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-18 02:59:00,731 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/nfs.dump.dir in system properties and HBase conf 2023-08-18 02:59:00,731 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/java.io.tmpdir in system properties and HBase conf 2023-08-18 02:59:00,731 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-18 02:59:00,731 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-18 02:59:00,731 INFO [Listener at localhost/37847] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-18 02:59:00,736 WARN [Listener at localhost/37847] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-18 02:59:00,736 WARN [Listener at localhost/37847] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-18 02:59:00,780 WARN [Listener at localhost/37847] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 02:59:00,782 INFO [Listener at localhost/37847] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 02:59:00,786 INFO [Listener at localhost/37847] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/java.io.tmpdir/Jetty_localhost_35785_hdfs____vqwn7g/webapp 2023-08-18 02:59:00,880 INFO [Listener at localhost/37847] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35785 2023-08-18 02:59:00,884 WARN [Listener at localhost/37847] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-18 02:59:00,885 WARN [Listener at localhost/37847] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-18 02:59:00,936 WARN [Listener at localhost/38247] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 02:59:00,951 WARN [Listener at localhost/38247] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-18 02:59:00,953 WARN [Listener at localhost/38247] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 02:59:00,954 INFO [Listener at localhost/38247] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 02:59:00,959 INFO [Listener at localhost/38247] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/java.io.tmpdir/Jetty_localhost_37373_datanode____4ecdzm/webapp 2023-08-18 02:59:01,055 INFO [Listener at localhost/38247] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37373 2023-08-18 02:59:01,061 WARN [Listener at localhost/42131] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 02:59:01,074 WARN [Listener at localhost/42131] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-18 02:59:01,076 WARN [Listener at localhost/42131] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 02:59:01,077 INFO [Listener at localhost/42131] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 02:59:01,080 INFO [Listener at localhost/42131] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/java.io.tmpdir/Jetty_localhost_42517_datanode____7jzt50/webapp 2023-08-18 02:59:01,169 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc5dcc9838471e85f: Processing first storage report for DS-89e5dd2c-0fdc-4f63-bd0c-aa5858d51fab from datanode c8d9ba24-6e75-4f54-82ad-000146c5bb61 2023-08-18 02:59:01,170 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc5dcc9838471e85f: from storage DS-89e5dd2c-0fdc-4f63-bd0c-aa5858d51fab node DatanodeRegistration(127.0.0.1:40861, datanodeUuid=c8d9ba24-6e75-4f54-82ad-000146c5bb61, infoPort=35411, infoSecurePort=0, ipcPort=42131, storageInfo=lv=-57;cid=testClusterID;nsid=18306714;c=1692327540739), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 02:59:01,170 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc5dcc9838471e85f: Processing first storage report for DS-56aee0b9-f4b0-44c9-bd1a-247d499c1522 from datanode c8d9ba24-6e75-4f54-82ad-000146c5bb61 2023-08-18 02:59:01,170 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc5dcc9838471e85f: from storage DS-56aee0b9-f4b0-44c9-bd1a-247d499c1522 node DatanodeRegistration(127.0.0.1:40861, datanodeUuid=c8d9ba24-6e75-4f54-82ad-000146c5bb61, infoPort=35411, infoSecurePort=0, ipcPort=42131, storageInfo=lv=-57;cid=testClusterID;nsid=18306714;c=1692327540739), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 02:59:01,186 INFO [Listener at localhost/42131] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42517 2023-08-18 02:59:01,193 WARN [Listener at localhost/34121] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 02:59:01,290 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3cd2d9176e82970c: Processing first storage report for DS-a4dae326-16bc-4b1c-aa6a-9e8385bab75f from datanode b45608b2-049f-44ec-b8c8-8d9fbd2bb658 2023-08-18 02:59:01,291 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3cd2d9176e82970c: from storage DS-a4dae326-16bc-4b1c-aa6a-9e8385bab75f node DatanodeRegistration(127.0.0.1:37287, datanodeUuid=b45608b2-049f-44ec-b8c8-8d9fbd2bb658, infoPort=40459, infoSecurePort=0, ipcPort=34121, storageInfo=lv=-57;cid=testClusterID;nsid=18306714;c=1692327540739), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 02:59:01,291 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3cd2d9176e82970c: Processing first storage report for DS-d1732d9b-3f10-44e8-9849-6b7ba46f8c4f from datanode b45608b2-049f-44ec-b8c8-8d9fbd2bb658 2023-08-18 02:59:01,291 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3cd2d9176e82970c: from storage DS-d1732d9b-3f10-44e8-9849-6b7ba46f8c4f node DatanodeRegistration(127.0.0.1:37287, datanodeUuid=b45608b2-049f-44ec-b8c8-8d9fbd2bb658, infoPort=40459, infoSecurePort=0, ipcPort=34121, storageInfo=lv=-57;cid=testClusterID;nsid=18306714;c=1692327540739), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 02:59:01,301 DEBUG [Listener at localhost/34121] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa 2023-08-18 02:59:01,305 INFO [Listener at localhost/34121] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/cluster_8224e203-f1af-f19e-05b6-812bd3cec471/zookeeper_0, clientPort=50119, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/cluster_8224e203-f1af-f19e-05b6-812bd3cec471/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/cluster_8224e203-f1af-f19e-05b6-812bd3cec471/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-18 02:59:01,306 INFO [Listener at localhost/34121] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=50119 2023-08-18 02:59:01,307 INFO [Listener at localhost/34121] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:59:01,308 INFO [Listener at localhost/34121] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:59:01,324 INFO [Listener at localhost/34121] util.FSUtils(471): Created version file at hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c with version=8 2023-08-18 02:59:01,324 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/hbase-staging 2023-08-18 02:59:01,326 INFO [Listener at localhost/34121] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-18 02:59:01,326 INFO [Listener at localhost/34121] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 02:59:01,326 INFO [Listener at localhost/34121] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-18 02:59:01,326 INFO [Listener at localhost/34121] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-18 02:59:01,326 INFO [Listener at localhost/34121] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 02:59:01,327 INFO [Listener at localhost/34121] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-18 02:59:01,327 INFO [Listener at localhost/34121] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-18 02:59:01,328 INFO [Listener at localhost/34121] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46191 2023-08-18 02:59:01,328 INFO [Listener at localhost/34121] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:59:01,330 INFO [Listener at localhost/34121] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:59:01,331 INFO [Listener at localhost/34121] zookeeper.RecoverableZooKeeper(93): Process identifier=master:46191 connecting to ZooKeeper ensemble=127.0.0.1:50119 2023-08-18 02:59:01,338 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:461910x0, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-18 02:59:01,339 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:46191-0x1001d5431de0000 connected 2023-08-18 02:59:01,357 DEBUG [Listener at localhost/34121] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-18 02:59:01,358 DEBUG [Listener at localhost/34121] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 02:59:01,358 DEBUG [Listener at localhost/34121] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-18 02:59:01,359 DEBUG [Listener at localhost/34121] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46191 2023-08-18 02:59:01,359 DEBUG [Listener at localhost/34121] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46191 2023-08-18 02:59:01,359 DEBUG [Listener at localhost/34121] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46191 2023-08-18 02:59:01,360 DEBUG [Listener at localhost/34121] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46191 2023-08-18 02:59:01,360 DEBUG [Listener at localhost/34121] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46191 2023-08-18 02:59:01,360 INFO [Listener at localhost/34121] master.HMaster(444): hbase.rootdir=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c, hbase.cluster.distributed=false 2023-08-18 02:59:01,373 INFO [Listener at localhost/34121] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-18 02:59:01,374 INFO [Listener at localhost/34121] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 02:59:01,374 INFO [Listener at localhost/34121] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-18 02:59:01,374 INFO [Listener at localhost/34121] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-18 02:59:01,374 INFO [Listener at localhost/34121] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 02:59:01,374 INFO [Listener at localhost/34121] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-18 02:59:01,374 INFO [Listener at localhost/34121] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-18 02:59:01,377 INFO [Listener at localhost/34121] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40589 2023-08-18 02:59:01,377 INFO [Listener at localhost/34121] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-18 02:59:01,378 DEBUG [Listener at localhost/34121] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-18 02:59:01,379 INFO [Listener at localhost/34121] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:59:01,380 INFO [Listener at localhost/34121] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:59:01,381 INFO [Listener at localhost/34121] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40589 connecting to ZooKeeper ensemble=127.0.0.1:50119 2023-08-18 02:59:01,384 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:405890x0, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-18 02:59:01,385 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:40589-0x1001d5431de0001 connected 2023-08-18 02:59:01,386 DEBUG [Listener at localhost/34121] zookeeper.ZKUtil(164): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-18 02:59:01,386 DEBUG [Listener at localhost/34121] zookeeper.ZKUtil(164): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 02:59:01,387 DEBUG [Listener at localhost/34121] zookeeper.ZKUtil(164): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-18 02:59:01,387 DEBUG [Listener at localhost/34121] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40589 2023-08-18 02:59:01,387 DEBUG [Listener at localhost/34121] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40589 2023-08-18 02:59:01,388 DEBUG [Listener at localhost/34121] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40589 2023-08-18 02:59:01,388 DEBUG [Listener at localhost/34121] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40589 2023-08-18 02:59:01,388 DEBUG [Listener at localhost/34121] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40589 2023-08-18 02:59:01,389 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,46191,1692327541325 2023-08-18 02:59:01,391 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-18 02:59:01,392 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,46191,1692327541325 2023-08-18 02:59:01,393 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-18 02:59:01,393 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-18 02:59:01,394 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:59:01,394 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-18 02:59:01,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-18 02:59:01,395 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,46191,1692327541325 from backup master directory 2023-08-18 02:59:01,398 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,46191,1692327541325 2023-08-18 02:59:01,398 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-18 02:59:01,398 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-18 02:59:01,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,46191,1692327541325 2023-08-18 02:59:01,416 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/hbase.id with ID: 65fc3d28-58cd-4e6b-a16f-b0cfe29aef1d 2023-08-18 02:59:01,432 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:59:01,435 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:59:01,446 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x7cd64736 to 127.0.0.1:50119 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-18 02:59:01,450 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@75995ba5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-18 02:59:01,450 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-18 02:59:01,451 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-18 02:59:01,451 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-18 02:59:01,452 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/MasterData/data/master/store-tmp 2023-08-18 02:59:01,463 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:59:01,463 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-18 02:59:01,463 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 02:59:01,464 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 02:59:01,464 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-18 02:59:01,464 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 02:59:01,464 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 02:59:01,464 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-18 02:59:01,464 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/MasterData/WALs/jenkins-hbase4.apache.org,46191,1692327541325 2023-08-18 02:59:01,468 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46191%2C1692327541325, suffix=, logDir=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/MasterData/WALs/jenkins-hbase4.apache.org,46191,1692327541325, archiveDir=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/MasterData/oldWALs, maxLogs=10 2023-08-18 02:59:01,484 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40861,DS-89e5dd2c-0fdc-4f63-bd0c-aa5858d51fab,DISK] 2023-08-18 02:59:01,484 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37287,DS-a4dae326-16bc-4b1c-aa6a-9e8385bab75f,DISK] 2023-08-18 02:59:01,487 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/MasterData/WALs/jenkins-hbase4.apache.org,46191,1692327541325/jenkins-hbase4.apache.org%2C46191%2C1692327541325.1692327541468 2023-08-18 02:59:01,487 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40861,DS-89e5dd2c-0fdc-4f63-bd0c-aa5858d51fab,DISK], DatanodeInfoWithStorage[127.0.0.1:37287,DS-a4dae326-16bc-4b1c-aa6a-9e8385bab75f,DISK]] 2023-08-18 02:59:01,487 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-18 02:59:01,487 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:59:01,487 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-18 02:59:01,487 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-18 02:59:01,490 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-18 02:59:01,491 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-18 02:59:01,492 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-18 02:59:01,493 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:59:01,494 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-18 02:59:01,495 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-18 02:59:01,499 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-18 02:59:01,502 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-18 02:59:01,503 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=826746, jitterRate=0.051262035965919495}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-18 02:59:01,503 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-18 02:59:01,504 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-18 02:59:01,505 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-18 02:59:01,505 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-18 02:59:01,505 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-18 02:59:01,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-18 02:59:01,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-08-18 02:59:01,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-18 02:59:01,509 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-18 02:59:01,511 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-18 02:59:01,521 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-18 02:59:01,522 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-18 02:59:01,522 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-18 02:59:01,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-18 02:59:01,523 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-18 02:59:01,525 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:59:01,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-18 02:59:01,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-18 02:59:01,528 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-18 02:59:01,531 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-18 02:59:01,531 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-18 02:59:01,531 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:59:01,532 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,46191,1692327541325, sessionid=0x1001d5431de0000, setting cluster-up flag (Was=false) 2023-08-18 02:59:01,535 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:59:01,541 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-18 02:59:01,542 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46191,1692327541325 2023-08-18 02:59:01,545 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:59:01,551 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-18 02:59:01,552 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46191,1692327541325 2023-08-18 02:59:01,553 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/.hbase-snapshot/.tmp 2023-08-18 02:59:01,556 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-18 02:59:01,556 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 02:59:01,556 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 02:59:01,556 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 02:59:01,556 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 02:59:01,556 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-18 02:59:01,556 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:59:01,556 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-18 02:59:01,556 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:59:01,558 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692327571557 2023-08-18 02:59:01,558 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-18 02:59:01,558 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-18 02:59:01,558 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-18 02:59:01,558 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-18 02:59:01,558 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-18 02:59:01,558 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-18 02:59:01,558 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:01,559 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-18 02:59:01,559 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-18 02:59:01,559 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-18 02:59:01,559 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-18 02:59:01,559 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-18 02:59:01,561 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-18 02:59:01,561 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-18 02:59:01,561 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-18 02:59:01,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692327541561,5,FailOnTimeoutGroup] 2023-08-18 02:59:01,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692327541561,5,FailOnTimeoutGroup] 2023-08-18 02:59:01,561 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:01,561 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-18 02:59:01,562 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:01,562 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:01,580 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-18 02:59:01,581 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-18 02:59:01,581 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c 2023-08-18 02:59:01,590 INFO [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(951): ClusterId : 65fc3d28-58cd-4e6b-a16f-b0cfe29aef1d 2023-08-18 02:59:01,591 DEBUG [RS:0;jenkins-hbase4:40589] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-18 02:59:01,592 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:59:01,594 DEBUG [RS:0;jenkins-hbase4:40589] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-18 02:59:01,594 DEBUG [RS:0;jenkins-hbase4:40589] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-18 02:59:01,594 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-18 02:59:01,596 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/info 2023-08-18 02:59:01,597 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-18 02:59:01,598 DEBUG [RS:0;jenkins-hbase4:40589] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-18 02:59:01,599 DEBUG [RS:0;jenkins-hbase4:40589] zookeeper.ReadOnlyZKClient(139): Connect 0x1fc2f7f8 to 127.0.0.1:50119 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-18 02:59:01,599 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:59:01,599 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-18 02:59:01,601 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/rep_barrier 2023-08-18 02:59:01,602 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-18 02:59:01,603 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:59:01,603 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-18 02:59:01,604 DEBUG [RS:0;jenkins-hbase4:40589] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@11051a70, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-18 02:59:01,605 DEBUG [RS:0;jenkins-hbase4:40589] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@153a9c88, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-18 02:59:01,606 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/table 2023-08-18 02:59:01,606 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-18 02:59:01,607 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:59:01,608 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740 2023-08-18 02:59:01,609 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740 2023-08-18 02:59:01,612 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-18 02:59:01,614 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-18 02:59:01,616 DEBUG [RS:0;jenkins-hbase4:40589] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:40589 2023-08-18 02:59:01,616 INFO [RS:0;jenkins-hbase4:40589] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-18 02:59:01,616 INFO [RS:0;jenkins-hbase4:40589] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-18 02:59:01,616 DEBUG [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(1022): About to register with Master. 2023-08-18 02:59:01,616 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-18 02:59:01,617 INFO [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,46191,1692327541325 with isa=jenkins-hbase4.apache.org/172.31.14.131:40589, startcode=1692327541373 2023-08-18 02:59:01,617 DEBUG [RS:0;jenkins-hbase4:40589] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-18 02:59:01,617 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=861912, jitterRate=0.09597845375537872}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-18 02:59:01,618 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-18 02:59:01,618 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-18 02:59:01,618 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-18 02:59:01,618 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-18 02:59:01,618 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-18 02:59:01,618 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-18 02:59:01,619 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-18 02:59:01,620 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-18 02:59:01,620 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39933, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-08-18 02:59:01,621 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=46191] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:01,622 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-18 02:59:01,622 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-18 02:59:01,622 DEBUG [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c 2023-08-18 02:59:01,622 DEBUG [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:38247 2023-08-18 02:59:01,622 DEBUG [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-18 02:59:01,622 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-18 02:59:01,624 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-18 02:59:01,624 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-18 02:59:01,626 DEBUG [RS:0;jenkins-hbase4:40589] zookeeper.ZKUtil(162): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:01,626 WARN [RS:0;jenkins-hbase4:40589] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-18 02:59:01,626 INFO [RS:0;jenkins-hbase4:40589] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-18 02:59:01,626 DEBUG [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(1948): logDir=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:01,626 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-18 02:59:01,626 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,40589,1692327541373] 2023-08-18 02:59:01,630 DEBUG [RS:0;jenkins-hbase4:40589] zookeeper.ZKUtil(162): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:01,631 DEBUG [RS:0;jenkins-hbase4:40589] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-18 02:59:01,632 INFO [RS:0;jenkins-hbase4:40589] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-18 02:59:01,633 INFO [RS:0;jenkins-hbase4:40589] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-18 02:59:01,634 INFO [RS:0;jenkins-hbase4:40589] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-18 02:59:01,634 INFO [RS:0;jenkins-hbase4:40589] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:01,634 INFO [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-18 02:59:01,635 INFO [RS:0;jenkins-hbase4:40589] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:01,635 DEBUG [RS:0;jenkins-hbase4:40589] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:59:01,635 DEBUG [RS:0;jenkins-hbase4:40589] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:59:01,635 DEBUG [RS:0;jenkins-hbase4:40589] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:59:01,635 DEBUG [RS:0;jenkins-hbase4:40589] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:59:01,636 DEBUG [RS:0;jenkins-hbase4:40589] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:59:01,636 DEBUG [RS:0;jenkins-hbase4:40589] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-18 02:59:01,636 DEBUG [RS:0;jenkins-hbase4:40589] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:59:01,636 DEBUG [RS:0;jenkins-hbase4:40589] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:59:01,636 DEBUG [RS:0;jenkins-hbase4:40589] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:59:01,636 DEBUG [RS:0;jenkins-hbase4:40589] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 02:59:01,637 INFO [RS:0;jenkins-hbase4:40589] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:01,637 INFO [RS:0;jenkins-hbase4:40589] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:01,637 INFO [RS:0;jenkins-hbase4:40589] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:01,651 INFO [RS:0;jenkins-hbase4:40589] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-18 02:59:01,651 INFO [RS:0;jenkins-hbase4:40589] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40589,1692327541373-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:01,668 INFO [RS:0;jenkins-hbase4:40589] regionserver.Replication(203): jenkins-hbase4.apache.org,40589,1692327541373 started 2023-08-18 02:59:01,668 INFO [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,40589,1692327541373, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:40589, sessionid=0x1001d5431de0001 2023-08-18 02:59:01,668 DEBUG [RS:0;jenkins-hbase4:40589] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-18 02:59:01,668 DEBUG [RS:0;jenkins-hbase4:40589] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:01,668 DEBUG [RS:0;jenkins-hbase4:40589] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40589,1692327541373' 2023-08-18 02:59:01,669 DEBUG [RS:0;jenkins-hbase4:40589] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-18 02:59:01,669 DEBUG [RS:0;jenkins-hbase4:40589] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-18 02:59:01,670 DEBUG [RS:0;jenkins-hbase4:40589] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-18 02:59:01,670 DEBUG [RS:0;jenkins-hbase4:40589] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-18 02:59:01,670 DEBUG [RS:0;jenkins-hbase4:40589] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:01,670 DEBUG [RS:0;jenkins-hbase4:40589] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40589,1692327541373' 2023-08-18 02:59:01,670 DEBUG [RS:0;jenkins-hbase4:40589] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-18 02:59:01,670 DEBUG [RS:0;jenkins-hbase4:40589] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-18 02:59:01,671 DEBUG [RS:0;jenkins-hbase4:40589] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-18 02:59:01,671 INFO [RS:0;jenkins-hbase4:40589] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-18 02:59:01,671 INFO [RS:0;jenkins-hbase4:40589] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-18 02:59:01,774 INFO [RS:0;jenkins-hbase4:40589] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40589%2C1692327541373, suffix=, logDir=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373, archiveDir=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/oldWALs, maxLogs=32 2023-08-18 02:59:01,776 DEBUG [jenkins-hbase4:46191] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-18 02:59:01,777 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40589,1692327541373, state=OPENING 2023-08-18 02:59:01,779 DEBUG [PEWorker-2] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-18 02:59:01,781 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:59:01,781 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-18 02:59:01,781 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40589,1692327541373}] 2023-08-18 02:59:01,794 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37287,DS-a4dae326-16bc-4b1c-aa6a-9e8385bab75f,DISK] 2023-08-18 02:59:01,794 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40861,DS-89e5dd2c-0fdc-4f63-bd0c-aa5858d51fab,DISK] 2023-08-18 02:59:01,797 INFO [RS:0;jenkins-hbase4:40589] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373/jenkins-hbase4.apache.org%2C40589%2C1692327541373.1692327541775 2023-08-18 02:59:01,797 DEBUG [RS:0;jenkins-hbase4:40589] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37287,DS-a4dae326-16bc-4b1c-aa6a-9e8385bab75f,DISK], DatanodeInfoWithStorage[127.0.0.1:40861,DS-89e5dd2c-0fdc-4f63-bd0c-aa5858d51fab,DISK]] 2023-08-18 02:59:01,937 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:01,937 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-18 02:59:01,939 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55914, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-18 02:59:01,945 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-18 02:59:01,945 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-18 02:59:01,947 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40589%2C1692327541373.meta, suffix=.meta, logDir=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373, archiveDir=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/oldWALs, maxLogs=32 2023-08-18 02:59:01,963 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40861,DS-89e5dd2c-0fdc-4f63-bd0c-aa5858d51fab,DISK] 2023-08-18 02:59:01,964 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37287,DS-a4dae326-16bc-4b1c-aa6a-9e8385bab75f,DISK] 2023-08-18 02:59:01,967 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373/jenkins-hbase4.apache.org%2C40589%2C1692327541373.meta.1692327541948.meta 2023-08-18 02:59:01,968 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40861,DS-89e5dd2c-0fdc-4f63-bd0c-aa5858d51fab,DISK], DatanodeInfoWithStorage[127.0.0.1:37287,DS-a4dae326-16bc-4b1c-aa6a-9e8385bab75f,DISK]] 2023-08-18 02:59:01,968 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-18 02:59:01,968 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-18 02:59:01,969 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-18 02:59:01,970 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-18 02:59:01,971 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-18 02:59:01,971 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:59:01,971 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-18 02:59:01,971 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-18 02:59:01,973 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-18 02:59:01,974 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/info 2023-08-18 02:59:01,974 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/info 2023-08-18 02:59:01,975 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-18 02:59:01,975 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:59:01,975 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-18 02:59:01,977 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/rep_barrier 2023-08-18 02:59:01,977 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/rep_barrier 2023-08-18 02:59:01,977 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-18 02:59:01,978 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:59:01,978 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-18 02:59:01,980 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/table 2023-08-18 02:59:01,981 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/table 2023-08-18 02:59:01,982 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-18 02:59:01,982 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:59:01,983 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740 2023-08-18 02:59:01,985 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740 2023-08-18 02:59:01,987 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-18 02:59:01,989 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-18 02:59:01,990 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=843136, jitterRate=0.07210315763950348}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-18 02:59:01,990 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-18 02:59:01,991 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1692327541937 2023-08-18 02:59:01,995 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-18 02:59:01,996 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-18 02:59:01,996 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40589,1692327541373, state=OPEN 2023-08-18 02:59:01,998 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-18 02:59:01,998 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-18 02:59:02,001 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-18 02:59:02,002 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40589,1692327541373 in 217 msec 2023-08-18 02:59:02,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-18 02:59:02,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 379 msec 2023-08-18 02:59:02,007 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 451 msec 2023-08-18 02:59:02,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1692327542007, completionTime=-1 2023-08-18 02:59:02,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-18 02:59:02,007 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-18 02:59:02,010 DEBUG [hconnection-0x55d0404-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-18 02:59:02,013 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55926, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-18 02:59:02,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-18 02:59:02,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692327602015 2023-08-18 02:59:02,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692327662015 2023-08-18 02:59:02,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 7 msec 2023-08-18 02:59:02,021 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46191,1692327541325-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:02,022 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46191,1692327541325-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:02,022 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46191,1692327541325-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:02,022 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:46191, period=300000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:02,022 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:02,022 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-18 02:59:02,022 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-18 02:59:02,023 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-18 02:59:02,024 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-18 02:59:02,025 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-18 02:59:02,026 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-18 02:59:02,028 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/.tmp/data/hbase/namespace/1ea064a6a9b903c05e42d398bfe357c7 2023-08-18 02:59:02,029 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/.tmp/data/hbase/namespace/1ea064a6a9b903c05e42d398bfe357c7 empty. 2023-08-18 02:59:02,030 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/.tmp/data/hbase/namespace/1ea064a6a9b903c05e42d398bfe357c7 2023-08-18 02:59:02,030 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-18 02:59:02,045 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-18 02:59:02,046 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1ea064a6a9b903c05e42d398bfe357c7, NAME => 'hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/.tmp 2023-08-18 02:59:02,059 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:59:02,059 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 1ea064a6a9b903c05e42d398bfe357c7, disabling compactions & flushes 2023-08-18 02:59:02,059 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. 2023-08-18 02:59:02,059 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. 2023-08-18 02:59:02,059 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. after waiting 0 ms 2023-08-18 02:59:02,059 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. 2023-08-18 02:59:02,059 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. 2023-08-18 02:59:02,059 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 1ea064a6a9b903c05e42d398bfe357c7: 2023-08-18 02:59:02,062 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-18 02:59:02,064 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692327542064"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692327542064"}]},"ts":"1692327542064"} 2023-08-18 02:59:02,067 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-18 02:59:02,068 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-18 02:59:02,068 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692327542068"}]},"ts":"1692327542068"} 2023-08-18 02:59:02,070 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-18 02:59:02,077 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=1ea064a6a9b903c05e42d398bfe357c7, ASSIGN}] 2023-08-18 02:59:02,080 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=1ea064a6a9b903c05e42d398bfe357c7, ASSIGN 2023-08-18 02:59:02,082 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=1ea064a6a9b903c05e42d398bfe357c7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40589,1692327541373; forceNewPlan=false, retain=false 2023-08-18 02:59:02,233 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=1ea064a6a9b903c05e42d398bfe357c7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:02,233 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692327542233"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692327542233"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692327542233"}]},"ts":"1692327542233"} 2023-08-18 02:59:02,236 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 1ea064a6a9b903c05e42d398bfe357c7, server=jenkins-hbase4.apache.org,40589,1692327541373}] 2023-08-18 02:59:02,394 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. 2023-08-18 02:59:02,394 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1ea064a6a9b903c05e42d398bfe357c7, NAME => 'hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7.', STARTKEY => '', ENDKEY => ''} 2023-08-18 02:59:02,395 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 1ea064a6a9b903c05e42d398bfe357c7 2023-08-18 02:59:02,395 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:59:02,395 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1ea064a6a9b903c05e42d398bfe357c7 2023-08-18 02:59:02,395 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1ea064a6a9b903c05e42d398bfe357c7 2023-08-18 02:59:02,397 INFO [StoreOpener-1ea064a6a9b903c05e42d398bfe357c7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1ea064a6a9b903c05e42d398bfe357c7 2023-08-18 02:59:02,399 DEBUG [StoreOpener-1ea064a6a9b903c05e42d398bfe357c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/namespace/1ea064a6a9b903c05e42d398bfe357c7/info 2023-08-18 02:59:02,399 DEBUG [StoreOpener-1ea064a6a9b903c05e42d398bfe357c7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/namespace/1ea064a6a9b903c05e42d398bfe357c7/info 2023-08-18 02:59:02,399 INFO [StoreOpener-1ea064a6a9b903c05e42d398bfe357c7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1ea064a6a9b903c05e42d398bfe357c7 columnFamilyName info 2023-08-18 02:59:02,400 INFO [StoreOpener-1ea064a6a9b903c05e42d398bfe357c7-1] regionserver.HStore(310): Store=1ea064a6a9b903c05e42d398bfe357c7/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:59:02,401 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/namespace/1ea064a6a9b903c05e42d398bfe357c7 2023-08-18 02:59:02,402 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/namespace/1ea064a6a9b903c05e42d398bfe357c7 2023-08-18 02:59:02,405 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1ea064a6a9b903c05e42d398bfe357c7 2023-08-18 02:59:02,407 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/namespace/1ea064a6a9b903c05e42d398bfe357c7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-18 02:59:02,408 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1ea064a6a9b903c05e42d398bfe357c7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=847890, jitterRate=0.07814808189868927}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-18 02:59:02,408 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1ea064a6a9b903c05e42d398bfe357c7: 2023-08-18 02:59:02,409 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7., pid=6, masterSystemTime=1692327542389 2023-08-18 02:59:02,412 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. 2023-08-18 02:59:02,412 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. 2023-08-18 02:59:02,413 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=1ea064a6a9b903c05e42d398bfe357c7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:02,413 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692327542413"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692327542413"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692327542413"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692327542413"}]},"ts":"1692327542413"} 2023-08-18 02:59:02,418 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-18 02:59:02,418 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 1ea064a6a9b903c05e42d398bfe357c7, server=jenkins-hbase4.apache.org,40589,1692327541373 in 180 msec 2023-08-18 02:59:02,421 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-18 02:59:02,422 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=1ea064a6a9b903c05e42d398bfe357c7, ASSIGN in 341 msec 2023-08-18 02:59:02,422 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-18 02:59:02,423 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692327542422"}]},"ts":"1692327542422"} 2023-08-18 02:59:02,425 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-18 02:59:02,427 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-18 02:59:02,427 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-18 02:59:02,428 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-18 02:59:02,428 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:59:02,430 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 405 msec 2023-08-18 02:59:02,433 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-18 02:59:02,444 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-18 02:59:02,448 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 14 msec 2023-08-18 02:59:02,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-18 02:59:02,464 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-18 02:59:02,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 13 msec 2023-08-18 02:59:02,481 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-18 02:59:02,484 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-18 02:59:02,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.085sec 2023-08-18 02:59:02,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-18 02:59:02,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-18 02:59:02,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-18 02:59:02,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46191,1692327541325-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-18 02:59:02,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46191,1692327541325-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-18 02:59:02,486 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-18 02:59:02,490 DEBUG [Listener at localhost/34121] zookeeper.ReadOnlyZKClient(139): Connect 0x0d7edd75 to 127.0.0.1:50119 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-18 02:59:02,497 DEBUG [Listener at localhost/34121] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@9b3403c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-18 02:59:02,498 DEBUG [hconnection-0x2f53368f-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-18 02:59:02,500 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55928, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-18 02:59:02,502 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,46191,1692327541325 2023-08-18 02:59:02,503 INFO [Listener at localhost/34121] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 02:59:02,506 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-18 02:59:02,506 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 02:59:02,507 INFO [Listener at localhost/34121] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-18 02:59:02,510 DEBUG [Listener at localhost/34121] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-18 02:59:02,513 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40380, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-18 02:59:02,515 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-08-18 02:59:02,515 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-08-18 02:59:02,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-18 02:59:02,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:02,518 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-08-18 02:59:02,518 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testCompactionRecordDoesntBlockRolling" procId is: 9 2023-08-18 02:59:02,519 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-18 02:59:02,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-18 02:59:02,522 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754 2023-08-18 02:59:02,522 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754 empty. 2023-08-18 02:59:02,523 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754 2023-08-18 02:59:02,523 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testCompactionRecordDoesntBlockRolling regions 2023-08-18 02:59:02,537 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/.tabledesc/.tableinfo.0000000001 2023-08-18 02:59:02,538 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => e3c356555f1d3f311cfa4b29c30ff754, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/.tmp 2023-08-18 02:59:02,548 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:59:02,548 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1604): Closing e3c356555f1d3f311cfa4b29c30ff754, disabling compactions & flushes 2023-08-18 02:59:02,548 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:02,548 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:02,548 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. after waiting 0 ms 2023-08-18 02:59:02,548 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:02,548 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:02,548 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1558): Region close journal for e3c356555f1d3f311cfa4b29c30ff754: 2023-08-18 02:59:02,551 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ADD_TO_META 2023-08-18 02:59:02,553 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1692327542552"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692327542552"}]},"ts":"1692327542552"} 2023-08-18 02:59:02,555 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-18 02:59:02,556 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-18 02:59:02,556 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692327542556"}]},"ts":"1692327542556"} 2023-08-18 02:59:02,558 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLING in hbase:meta 2023-08-18 02:59:02,563 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=e3c356555f1d3f311cfa4b29c30ff754, ASSIGN}] 2023-08-18 02:59:02,565 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=e3c356555f1d3f311cfa4b29c30ff754, ASSIGN 2023-08-18 02:59:02,566 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=e3c356555f1d3f311cfa4b29c30ff754, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40589,1692327541373; forceNewPlan=false, retain=false 2023-08-18 02:59:02,717 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=e3c356555f1d3f311cfa4b29c30ff754, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:02,718 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1692327542717"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692327542717"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692327542717"}]},"ts":"1692327542717"} 2023-08-18 02:59:02,720 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure e3c356555f1d3f311cfa4b29c30ff754, server=jenkins-hbase4.apache.org,40589,1692327541373}] 2023-08-18 02:59:02,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:02,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e3c356555f1d3f311cfa4b29c30ff754, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754.', STARTKEY => '', ENDKEY => ''} 2023-08-18 02:59:02,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testCompactionRecordDoesntBlockRolling e3c356555f1d3f311cfa4b29c30ff754 2023-08-18 02:59:02,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 02:59:02,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e3c356555f1d3f311cfa4b29c30ff754 2023-08-18 02:59:02,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e3c356555f1d3f311cfa4b29c30ff754 2023-08-18 02:59:02,881 INFO [StoreOpener-e3c356555f1d3f311cfa4b29c30ff754-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region e3c356555f1d3f311cfa4b29c30ff754 2023-08-18 02:59:02,883 DEBUG [StoreOpener-e3c356555f1d3f311cfa4b29c30ff754-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info 2023-08-18 02:59:02,883 DEBUG [StoreOpener-e3c356555f1d3f311cfa4b29c30ff754-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info 2023-08-18 02:59:02,883 INFO [StoreOpener-e3c356555f1d3f311cfa4b29c30ff754-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e3c356555f1d3f311cfa4b29c30ff754 columnFamilyName info 2023-08-18 02:59:02,884 INFO [StoreOpener-e3c356555f1d3f311cfa4b29c30ff754-1] regionserver.HStore(310): Store=e3c356555f1d3f311cfa4b29c30ff754/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 02:59:02,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754 2023-08-18 02:59:02,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754 2023-08-18 02:59:02,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e3c356555f1d3f311cfa4b29c30ff754 2023-08-18 02:59:02,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-18 02:59:02,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e3c356555f1d3f311cfa4b29c30ff754; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=711368, jitterRate=-0.09544909000396729}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-18 02:59:02,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e3c356555f1d3f311cfa4b29c30ff754: 2023-08-18 02:59:02,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754., pid=11, masterSystemTime=1692327542873 2023-08-18 02:59:02,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:02,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:02,896 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=e3c356555f1d3f311cfa4b29c30ff754, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:02,896 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1692327542896"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692327542896"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692327542896"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692327542896"}]},"ts":"1692327542896"} 2023-08-18 02:59:02,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-18 02:59:02,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure e3c356555f1d3f311cfa4b29c30ff754, server=jenkins-hbase4.apache.org,40589,1692327541373 in 179 msec 2023-08-18 02:59:02,903 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-18 02:59:02,904 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=e3c356555f1d3f311cfa4b29c30ff754, ASSIGN in 338 msec 2023-08-18 02:59:02,905 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-18 02:59:02,905 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692327542905"}]},"ts":"1692327542905"} 2023-08-18 02:59:02,907 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLED in hbase:meta 2023-08-18 02:59:02,909 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_POST_OPERATION 2023-08-18 02:59:02,911 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling in 394 msec 2023-08-18 02:59:03,044 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-18 02:59:06,991 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-18 02:59:06,991 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-08-18 02:59:07,632 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-18 02:59:07,632 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-18 02:59:07,633 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-18 02:59:08,382 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-18 02:59:12,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-18 02:59:12,522 INFO [Listener at localhost/34121] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testCompactionRecordDoesntBlockRolling, procId: 9 completed 2023-08-18 02:59:12,526 DEBUG [Listener at localhost/34121] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:12,526 DEBUG [Listener at localhost/34121] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:12,540 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-18 02:59:12,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] procedure.ProcedureCoordinator(165): Submitting procedure hbase:namespace 2023-08-18 02:59:12,547 INFO [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'hbase:namespace' 2023-08-18 02:59:12,547 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-18 02:59:12,547 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'hbase:namespace' starting 'acquire' 2023-08-18 02:59:12,547 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'hbase:namespace', kicking off acquire phase on members. 2023-08-18 02:59:12,548 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-08-18 02:59:12,548 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-18 02:59:12,549 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-18 02:59:12,549 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-18 02:59:12,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-18 02:59:12,550 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,550 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-18 02:59:12,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/hbase:namespace 2023-08-18 02:59:12,550 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-08-18 02:59:12,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-18 02:59:12,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-18 02:59:12,551 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for hbase:namespace 2023-08-18 02:59:12,553 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:hbase:namespace 2023-08-18 02:59:12,553 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'hbase:namespace' with timeout 60000ms 2023-08-18 02:59:12,553 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-18 02:59:12,554 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'hbase:namespace' starting 'acquire' stage 2023-08-18 02:59:12,554 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-18 02:59:12,554 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-18 02:59:12,554 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. 2023-08-18 02:59:12,554 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. started... 2023-08-18 02:59:12,555 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1ea064a6a9b903c05e42d398bfe357c7 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-18 02:59:12,579 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/namespace/1ea064a6a9b903c05e42d398bfe357c7/.tmp/info/b5d9c5ac901e43dba079b7311456f7d7 2023-08-18 02:59:12,592 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/namespace/1ea064a6a9b903c05e42d398bfe357c7/.tmp/info/b5d9c5ac901e43dba079b7311456f7d7 as hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/namespace/1ea064a6a9b903c05e42d398bfe357c7/info/b5d9c5ac901e43dba079b7311456f7d7 2023-08-18 02:59:12,600 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/namespace/1ea064a6a9b903c05e42d398bfe357c7/info/b5d9c5ac901e43dba079b7311456f7d7, entries=2, sequenceid=6, filesize=4.8 K 2023-08-18 02:59:12,602 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 1ea064a6a9b903c05e42d398bfe357c7 in 47ms, sequenceid=6, compaction requested=false 2023-08-18 02:59:12,603 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1ea064a6a9b903c05e42d398bfe357c7: 2023-08-18 02:59:12,603 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. 2023-08-18 02:59:12,603 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-18 02:59:12,603 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-18 02:59:12,603 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,603 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'hbase:namespace' locally acquired 2023-08-18 02:59:12,603 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40589,1692327541373' joining acquired barrier for procedure (hbase:namespace) in zk 2023-08-18 02:59:12,605 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,605 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-18 02:59:12,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-18 02:59:12,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-18 02:59:12,606 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace 2023-08-18 02:59:12,606 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'hbase:namespace' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-18 02:59:12,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-18 02:59:12,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-18 02:59:12,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-18 02:59:12,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-18 02:59:12,608 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40589,1692327541373' joining acquired barrier for procedure 'hbase:namespace' on coordinator 2023-08-18 02:59:12,608 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'hbase:namespace' starting 'in-barrier' execution. 2023-08-18 02:59:12,608 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48bdf24c[Count = 0] remaining members to acquire global barrier 2023-08-18 02:59:12,608 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-18 02:59:12,610 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-08-18 02:59:12,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-18 02:59:12,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-18 02:59:12,610 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'hbase:namespace' received 'reached' from coordinator. 2023-08-18 02:59:12,610 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'hbase:namespace' locally completed 2023-08-18 02:59:12,610 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'hbase:namespace' completed for member 'jenkins-hbase4.apache.org,40589,1692327541373' in zk 2023-08-18 02:59:12,611 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,611 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-18 02:59:12,613 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'hbase:namespace' has notified controller of completion 2023-08-18 02:59:12,613 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,613 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-18 02:59:12,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-18 02:59:12,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-18 02:59:12,613 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'hbase:namespace' completed. 2023-08-18 02:59:12,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-18 02:59:12,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-18 02:59:12,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-18 02:59:12,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-18 02:59:12,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-18 02:59:12,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'hbase:namespace' member 'jenkins-hbase4.apache.org,40589,1692327541373': 2023-08-18 02:59:12,617 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40589,1692327541373' released barrier for procedure'hbase:namespace', counting down latch. Waiting for 0 more 2023-08-18 02:59:12,617 INFO [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'hbase:namespace' execution completed 2023-08-18 02:59:12,617 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-18 02:59:12,617 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-18 02:59:12,617 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:hbase:namespace 2023-08-18 02:59:12,617 INFO [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure hbase:namespaceincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-18 02:59:12,619 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-08-18 02:59:12,619 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-08-18 02:59:12,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/hbase:namespace 2023-08-18 02:59:12,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/hbase:namespace 2023-08-18 02:59:12,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-18 02:59:12,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-18 02:59:12,619 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-18 02:59:12,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-08-18 02:59:12,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-18 02:59:12,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-18 02:59:12,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-18 02:59:12,620 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-18 02:59:12,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-08-18 02:59:12,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-18 02:59:12,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-18 02:59:12,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,622 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-18 02:59:12,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-18 02:59:12,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,628 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,628 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-18 02:59:12,628 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-18 02:59:12,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-18 02:59:12,629 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-18 02:59:12,629 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-18 02:59:12,629 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:12,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-18 02:59:12,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'hbase:namespace' 2023-08-18 02:59:12,629 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-18 02:59:12,629 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-08-18 02:59:12,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-18 02:59:12,630 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-08-18 02:59:12,630 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-08-18 02:59:12,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-18 02:59:12,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-18 02:59:12,632 DEBUG [Listener at localhost/34121] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : hbase:namespace'' to complete. (max 20000 ms per retry) 2023-08-18 02:59:12,632 DEBUG [Listener at localhost/34121] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-18 02:59:22,632 DEBUG [Listener at localhost/34121] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-18 02:59:22,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-18 02:59:22,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-18 02:59:22,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,653 INFO [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-18 02:59:22,653 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-18 02:59:22,654 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-18 02:59:22,654 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-18 02:59:22,654 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,654 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,656 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-18 02:59:22,656 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-18 02:59:22,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-18 02:59:22,656 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,656 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-18 02:59:22,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,657 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-18 02:59:22,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,657 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,657 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,658 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-18 02:59:22,658 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-18 02:59:22,658 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-18 02:59:22,658 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-18 02:59:22,658 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-18 02:59:22,658 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:22,658 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. started... 2023-08-18 02:59:22,659 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e3c356555f1d3f311cfa4b29c30ff754 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-18 02:59:22,676 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=5 (bloomFilter=true), to=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/.tmp/info/e530ce283c064985b1faef78334bb020 2023-08-18 02:59:22,686 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/.tmp/info/e530ce283c064985b1faef78334bb020 as hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/e530ce283c064985b1faef78334bb020 2023-08-18 02:59:22,695 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/e530ce283c064985b1faef78334bb020, entries=1, sequenceid=5, filesize=5.8 K 2023-08-18 02:59:22,697 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for e3c356555f1d3f311cfa4b29c30ff754 in 38ms, sequenceid=5, compaction requested=false 2023-08-18 02:59:22,697 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e3c356555f1d3f311cfa4b29c30ff754: 2023-08-18 02:59:22,697 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:22,697 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-18 02:59:22,698 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-18 02:59:22,698 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,698 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-18 02:59:22,698 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40589,1692327541373' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-18 02:59:22,700 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,700 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-18 02:59:22,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-18 02:59:22,701 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,701 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-18 02:59:22,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-18 02:59:22,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-18 02:59:22,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-18 02:59:22,702 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40589,1692327541373' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-18 02:59:22,702 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c826f83[Count = 0] remaining members to acquire global barrier 2023-08-18 02:59:22,702 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-18 02:59:22,703 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,704 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,704 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-18 02:59:22,704 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-18 02:59:22,704 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,40589,1692327541373' in zk 2023-08-18 02:59:22,704 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,704 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-18 02:59:22,706 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-18 02:59:22,706 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,706 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-18 02:59:22,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-18 02:59:22,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-18 02:59:22,706 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-18 02:59:22,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-18 02:59:22,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-18 02:59:22,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-18 02:59:22,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,40589,1692327541373': 2023-08-18 02:59:22,709 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40589,1692327541373' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-18 02:59:22,709 INFO [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-18 02:59:22,709 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-18 02:59:22,709 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-18 02:59:22,709 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,709 INFO [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-18 02:59:22,714 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,714 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-18 02:59:22,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,714 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-18 02:59:22,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-18 02:59:22,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-18 02:59:22,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-18 02:59:22,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-18 02:59:22,715 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-18 02:59:22,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,716 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-18 02:59:22,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,719 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,719 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-18 02:59:22,719 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-18 02:59:22,720 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-18 02:59:22,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-18 02:59:22,720 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-18 02:59:22,720 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-18 02:59:22,720 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:22,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-18 02:59:22,720 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-08-18 02:59:22,721 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,721 DEBUG [Listener at localhost/34121] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-18 02:59:22,721 DEBUG [Listener at localhost/34121] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-18 02:59:22,721 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:22,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-18 02:59:22,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-18 02:59:22,721 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,721 DEBUG [Listener at localhost/34121] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-18 02:59:32,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-18 02:59:32,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-18 02:59:32,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-08-18 02:59:32,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,733 INFO [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-18 02:59:32,733 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-18 02:59:32,734 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-18 02:59:32,734 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-18 02:59:32,735 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,735 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,736 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,736 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-18 02:59:32,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-18 02:59:32,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-18 02:59:32,737 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,737 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-18 02:59:32,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,737 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-18 02:59:32,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,737 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,738 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-08-18 02:59:32,738 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,738 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-18 02:59:32,738 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-18 02:59:32,738 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-18 02:59:32,739 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-18 02:59:32,739 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-18 02:59:32,739 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:32,739 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. started... 2023-08-18 02:59:32,739 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e3c356555f1d3f311cfa4b29c30ff754 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-18 02:59:32,754 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=9 (bloomFilter=true), to=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/.tmp/info/fa81d69b1daf4edda044d8cee9009b16 2023-08-18 02:59:32,765 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/.tmp/info/fa81d69b1daf4edda044d8cee9009b16 as hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/fa81d69b1daf4edda044d8cee9009b16 2023-08-18 02:59:32,772 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/fa81d69b1daf4edda044d8cee9009b16, entries=1, sequenceid=9, filesize=5.8 K 2023-08-18 02:59:32,773 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for e3c356555f1d3f311cfa4b29c30ff754 in 34ms, sequenceid=9, compaction requested=false 2023-08-18 02:59:32,774 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e3c356555f1d3f311cfa4b29c30ff754: 2023-08-18 02:59:32,774 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:32,774 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-18 02:59:32,774 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-18 02:59:32,774 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,774 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-18 02:59:32,774 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40589,1692327541373' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-18 02:59:32,777 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,777 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-18 02:59:32,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-18 02:59:32,778 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-18 02:59:32,778 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-18 02:59:32,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-18 02:59:32,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-18 02:59:32,780 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40589,1692327541373' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-18 02:59:32,780 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@980f877[Count = 0] remaining members to acquire global barrier 2023-08-18 02:59:32,780 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-18 02:59:32,780 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,781 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,781 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-18 02:59:32,781 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-18 02:59:32,781 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,40589,1692327541373' in zk 2023-08-18 02:59:32,781 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,781 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-18 02:59:32,783 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-18 02:59:32,783 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,783 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-18 02:59:32,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,783 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-18 02:59:32,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-18 02:59:32,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-18 02:59:32,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-18 02:59:32,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-18 02:59:32,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-18 02:59:32,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,40589,1692327541373': 2023-08-18 02:59:32,786 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40589,1692327541373' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-18 02:59:32,786 INFO [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-18 02:59:32,786 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-18 02:59:32,786 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-18 02:59:32,786 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,786 INFO [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-18 02:59:32,788 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,788 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-18 02:59:32,788 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-18 02:59:32,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-18 02:59:32,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-18 02:59:32,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-18 02:59:32,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-18 02:59:32,789 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-18 02:59:32,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,790 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-18 02:59:32,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,794 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,794 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-18 02:59:32,794 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-18 02:59:32,794 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-18 02:59:32,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-08-18 02:59:32,794 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,794 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-18 02:59:32,795 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:32,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-18 02:59:32,795 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-18 02:59:32,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-18 02:59:32,795 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,795 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,795 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:32,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-18 02:59:32,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-18 02:59:32,796 DEBUG [Listener at localhost/34121] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-18 02:59:32,796 DEBUG [Listener at localhost/34121] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-18 02:59:42,796 DEBUG [Listener at localhost/34121] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-18 02:59:42,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-18 02:59:42,817 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37287,DS-a4dae326-16bc-4b1c-aa6a-9e8385bab75f,DISK] 2023-08-18 02:59:42,818 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40861,DS-89e5dd2c-0fdc-4f63-bd0c-aa5858d51fab,DISK] 2023-08-18 02:59:42,820 INFO [Listener at localhost/34121] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373/jenkins-hbase4.apache.org%2C40589%2C1692327541373.1692327541775 with entries=13, filesize=6.36 KB; new WAL /user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373/jenkins-hbase4.apache.org%2C40589%2C1692327541373.1692327582800 2023-08-18 02:59:42,821 DEBUG [Listener at localhost/34121] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37287,DS-a4dae326-16bc-4b1c-aa6a-9e8385bab75f,DISK], DatanodeInfoWithStorage[127.0.0.1:40861,DS-89e5dd2c-0fdc-4f63-bd0c-aa5858d51fab,DISK]] 2023-08-18 02:59:42,821 DEBUG [Listener at localhost/34121] wal.AbstractFSWAL(716): hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373/jenkins-hbase4.apache.org%2C40589%2C1692327541373.1692327541775 is not closed yet, will try archiving it next time 2023-08-18 02:59:42,829 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-18 02:59:42,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-08-18 02:59:42,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,832 INFO [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-18 02:59:42,832 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-18 02:59:42,833 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-18 02:59:42,833 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-18 02:59:42,833 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,833 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,835 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-18 02:59:42,835 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-18 02:59:42,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-18 02:59:42,835 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,835 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-18 02:59:42,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,836 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-18 02:59:42,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,836 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,836 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-08-18 02:59:42,836 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,836 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-18 02:59:42,836 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-18 02:59:42,837 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-18 02:59:42,837 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-18 02:59:42,837 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-18 02:59:42,837 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:42,837 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. started... 2023-08-18 02:59:42,837 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e3c356555f1d3f311cfa4b29c30ff754 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-18 02:59:42,851 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=13 (bloomFilter=true), to=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/.tmp/info/b9c84b180a374c54924745b3312c2b78 2023-08-18 02:59:42,860 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/.tmp/info/b9c84b180a374c54924745b3312c2b78 as hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/b9c84b180a374c54924745b3312c2b78 2023-08-18 02:59:42,867 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/b9c84b180a374c54924745b3312c2b78, entries=1, sequenceid=13, filesize=5.8 K 2023-08-18 02:59:42,868 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for e3c356555f1d3f311cfa4b29c30ff754 in 31ms, sequenceid=13, compaction requested=true 2023-08-18 02:59:42,868 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e3c356555f1d3f311cfa4b29c30ff754: 2023-08-18 02:59:42,868 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:42,868 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-18 02:59:42,869 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-18 02:59:42,869 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,869 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-18 02:59:42,869 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40589,1692327541373' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-18 02:59:42,871 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,871 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-18 02:59:42,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-18 02:59:42,871 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,871 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-18 02:59:42,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-18 02:59:42,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-18 02:59:42,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-18 02:59:42,873 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40589,1692327541373' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-18 02:59:42,873 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f4b2828[Count = 0] remaining members to acquire global barrier 2023-08-18 02:59:42,873 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-18 02:59:42,873 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,874 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,874 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-18 02:59:42,874 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-18 02:59:42,874 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,40589,1692327541373' in zk 2023-08-18 02:59:42,874 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,875 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-18 02:59:42,877 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,877 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-18 02:59:42,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-18 02:59:42,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-18 02:59:42,877 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-18 02:59:42,877 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-18 02:59:42,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-18 02:59:42,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-18 02:59:42,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-18 02:59:42,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,40589,1692327541373': 2023-08-18 02:59:42,880 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40589,1692327541373' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-18 02:59:42,880 INFO [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-18 02:59:42,880 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-18 02:59:42,880 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-18 02:59:42,880 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,880 INFO [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-18 02:59:42,882 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,882 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,882 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-18 02:59:42,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-18 02:59:42,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-18 02:59:42,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-18 02:59:42,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-18 02:59:42,883 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-18 02:59:42,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-18 02:59:42,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,884 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-18 02:59:42,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,886 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,886 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,886 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-18 02:59:42,886 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,887 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-18 02:59:42,887 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:42,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-18 02:59:42,887 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-18 02:59:42,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-18 02:59:42,887 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-18 02:59:42,887 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-18 02:59:42,888 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:42,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-18 02:59:42,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-18 02:59:42,888 DEBUG [Listener at localhost/34121] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-18 02:59:42,888 DEBUG [Listener at localhost/34121] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-18 02:59:52,888 DEBUG [Listener at localhost/34121] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-18 02:59:52,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-18 02:59:52,891 DEBUG [Listener at localhost/34121] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-18 02:59:52,896 DEBUG [Listener at localhost/34121] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 17694 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-18 02:59:52,898 DEBUG [Listener at localhost/34121] regionserver.HStore(1912): e3c356555f1d3f311cfa4b29c30ff754/info is initiating minor compaction (all files) 2023-08-18 02:59:52,898 INFO [Listener at localhost/34121] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-18 02:59:52,898 INFO [Listener at localhost/34121] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-18 02:59:52,899 INFO [Listener at localhost/34121] regionserver.HRegion(2259): Starting compaction of e3c356555f1d3f311cfa4b29c30ff754/info in TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:52,899 INFO [Listener at localhost/34121] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/e530ce283c064985b1faef78334bb020, hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/fa81d69b1daf4edda044d8cee9009b16, hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/b9c84b180a374c54924745b3312c2b78] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/.tmp, totalSize=17.3 K 2023-08-18 02:59:52,900 DEBUG [Listener at localhost/34121] compactions.Compactor(207): Compacting e530ce283c064985b1faef78334bb020, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1692327562644 2023-08-18 02:59:52,900 DEBUG [Listener at localhost/34121] compactions.Compactor(207): Compacting fa81d69b1daf4edda044d8cee9009b16, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1692327572724 2023-08-18 02:59:52,901 DEBUG [Listener at localhost/34121] compactions.Compactor(207): Compacting b9c84b180a374c54924745b3312c2b78, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1692327582799 2023-08-18 02:59:52,922 INFO [Listener at localhost/34121] throttle.PressureAwareThroughputController(145): e3c356555f1d3f311cfa4b29c30ff754#info#compaction#9 average throughput is 3.01 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-18 02:59:52,943 DEBUG [Listener at localhost/34121] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/.tmp/info/123f876a42b0433ebba2f8a98703f8a7 as hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/123f876a42b0433ebba2f8a98703f8a7 2023-08-18 02:59:52,958 INFO [Listener at localhost/34121] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in e3c356555f1d3f311cfa4b29c30ff754/info of e3c356555f1d3f311cfa4b29c30ff754 into 123f876a42b0433ebba2f8a98703f8a7(size=7.9 K), total size for store is 7.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-18 02:59:52,959 DEBUG [Listener at localhost/34121] regionserver.HRegion(2289): Compaction status journal for e3c356555f1d3f311cfa4b29c30ff754: 2023-08-18 02:59:52,975 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40861,DS-89e5dd2c-0fdc-4f63-bd0c-aa5858d51fab,DISK] 2023-08-18 02:59:52,976 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37287,DS-a4dae326-16bc-4b1c-aa6a-9e8385bab75f,DISK] 2023-08-18 02:59:52,979 INFO [Listener at localhost/34121] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373/jenkins-hbase4.apache.org%2C40589%2C1692327541373.1692327582800 with entries=4, filesize=2.43 KB; new WAL /user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373/jenkins-hbase4.apache.org%2C40589%2C1692327541373.1692327592961 2023-08-18 02:59:52,979 DEBUG [Listener at localhost/34121] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40861,DS-89e5dd2c-0fdc-4f63-bd0c-aa5858d51fab,DISK], DatanodeInfoWithStorage[127.0.0.1:37287,DS-a4dae326-16bc-4b1c-aa6a-9e8385bab75f,DISK]] 2023-08-18 02:59:52,980 DEBUG [Listener at localhost/34121] wal.AbstractFSWAL(716): hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373/jenkins-hbase4.apache.org%2C40589%2C1692327541373.1692327582800 is not closed yet, will try archiving it next time 2023-08-18 02:59:52,980 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373/jenkins-hbase4.apache.org%2C40589%2C1692327541373.1692327541775 to hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/oldWALs/jenkins-hbase4.apache.org%2C40589%2C1692327541373.1692327541775 2023-08-18 02:59:52,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-18 02:59:52,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-08-18 02:59:52,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:52,989 INFO [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-18 02:59:52,989 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-18 02:59:52,989 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-18 02:59:52,989 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-18 02:59:52,990 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:52,990 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:52,995 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:52,995 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-18 02:59:52,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-18 02:59:52,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-18 02:59:52,995 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:52,995 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-18 02:59:52,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:52,996 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:52,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-18 02:59:52,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:52,996 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:52,996 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-08-18 02:59:52,996 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:52,997 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-18 02:59:52,997 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-18 02:59:52,997 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-18 02:59:52,997 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-18 02:59:52,997 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-18 02:59:52,997 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:52,997 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. started... 2023-08-18 02:59:52,998 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e3c356555f1d3f311cfa4b29c30ff754 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-18 02:59:53,013 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=18 (bloomFilter=true), to=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/.tmp/info/2325a031b6864bb791c3b69acd6460a6 2023-08-18 02:59:53,022 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/.tmp/info/2325a031b6864bb791c3b69acd6460a6 as hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/2325a031b6864bb791c3b69acd6460a6 2023-08-18 02:59:53,029 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/2325a031b6864bb791c3b69acd6460a6, entries=1, sequenceid=18, filesize=5.8 K 2023-08-18 02:59:53,030 INFO [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for e3c356555f1d3f311cfa4b29c30ff754 in 33ms, sequenceid=18, compaction requested=false 2023-08-18 02:59:53,030 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e3c356555f1d3f311cfa4b29c30ff754: 2023-08-18 02:59:53,030 DEBUG [rs(jenkins-hbase4.apache.org,40589,1692327541373)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 02:59:53,030 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-18 02:59:53,031 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-18 02:59:53,031 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:53,031 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-18 02:59:53,031 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40589,1692327541373' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-18 02:59:53,033 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:53,033 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-18 02:59:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-18 02:59:53,033 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,033 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-18 02:59:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-18 02:59:53,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-18 02:59:53,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:53,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-18 02:59:53,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40589,1692327541373' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-18 02:59:53,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3843961f[Count = 0] remaining members to acquire global barrier 2023-08-18 02:59:53,035 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-18 02:59:53,035 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,036 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,037 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-18 02:59:53,037 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-18 02:59:53,037 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,40589,1692327541373' in zk 2023-08-18 02:59:53,037 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:53,037 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-18 02:59:53,038 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:53,038 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-18 02:59:53,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:53,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-18 02:59:53,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-18 02:59:53,038 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-18 02:59:53,039 DEBUG [member: 'jenkins-hbase4.apache.org,40589,1692327541373' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-18 02:59:53,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-18 02:59:53,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-18 02:59:53,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:53,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-18 02:59:53,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:53,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,40589,1692327541373': 2023-08-18 02:59:53,042 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40589,1692327541373' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-18 02:59:53,042 INFO [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-18 02:59:53,042 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-18 02:59:53,042 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-18 02:59:53,042 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,042 INFO [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-18 02:59:53,047 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,047 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-18 02:59:53,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-18 02:59:53,047 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-18 02:59:53,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-18 02:59:53,048 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:53,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-18 02:59:53,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-18 02:59:53,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-18 02:59:53,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,050 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:53,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:53,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-18 02:59:53,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:53,052 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-18 02:59:53,053 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-18 02:59:53,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-18 02:59:53,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-18 02:59:53,053 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:53,053 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,053 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,053 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 02:59:53,053 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-18 02:59:53,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-18 02:59:53,053 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,053 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-08-18 02:59:53,053 DEBUG [(jenkins-hbase4.apache.org,46191,1692327541325)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-18 02:59:53,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-18 02:59:53,054 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-18 02:59:53,053 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-18 02:59:53,055 DEBUG [Listener at localhost/34121] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-18 02:59:53,055 DEBUG [Listener at localhost/34121] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-18 03:00:03,055 DEBUG [Listener at localhost/34121] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-18 03:00:03,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46191] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-18 03:00:03,076 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37287,DS-a4dae326-16bc-4b1c-aa6a-9e8385bab75f,DISK] 2023-08-18 03:00:03,076 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40861,DS-89e5dd2c-0fdc-4f63-bd0c-aa5858d51fab,DISK] 2023-08-18 03:00:03,078 INFO [Listener at localhost/34121] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373/jenkins-hbase4.apache.org%2C40589%2C1692327541373.1692327592961 with entries=3, filesize=1.94 KB; new WAL /user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373/jenkins-hbase4.apache.org%2C40589%2C1692327541373.1692327603060 2023-08-18 03:00:03,078 DEBUG [Listener at localhost/34121] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40861,DS-89e5dd2c-0fdc-4f63-bd0c-aa5858d51fab,DISK], DatanodeInfoWithStorage[127.0.0.1:37287,DS-a4dae326-16bc-4b1c-aa6a-9e8385bab75f,DISK]] 2023-08-18 03:00:03,078 DEBUG [Listener at localhost/34121] wal.AbstractFSWAL(716): hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373/jenkins-hbase4.apache.org%2C40589%2C1692327541373.1692327592961 is not closed yet, will try archiving it next time 2023-08-18 03:00:03,079 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/WALs/jenkins-hbase4.apache.org,40589,1692327541373/jenkins-hbase4.apache.org%2C40589%2C1692327541373.1692327582800 to hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/oldWALs/jenkins-hbase4.apache.org%2C40589%2C1692327541373.1692327582800 2023-08-18 03:00:03,079 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-18 03:00:03,079 INFO [Listener at localhost/34121] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-18 03:00:03,079 DEBUG [Listener at localhost/34121] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0d7edd75 to 127.0.0.1:50119 2023-08-18 03:00:03,080 DEBUG [Listener at localhost/34121] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 03:00:03,080 DEBUG [Listener at localhost/34121] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-18 03:00:03,080 DEBUG [Listener at localhost/34121] util.JVMClusterUtil(257): Found active master hash=135988028, stopped=false 2023-08-18 03:00:03,080 INFO [Listener at localhost/34121] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,46191,1692327541325 2023-08-18 03:00:03,082 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-18 03:00:03,082 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-18 03:00:03,082 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:00:03,083 INFO [Listener at localhost/34121] procedure2.ProcedureExecutor(629): Stopping 2023-08-18 03:00:03,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 03:00:03,083 DEBUG [Listener at localhost/34121] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7cd64736 to 127.0.0.1:50119 2023-08-18 03:00:03,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 03:00:03,083 DEBUG [Listener at localhost/34121] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 03:00:03,084 INFO [Listener at localhost/34121] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40589,1692327541373' ***** 2023-08-18 03:00:03,084 INFO [Listener at localhost/34121] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-18 03:00:03,084 INFO [RS:0;jenkins-hbase4:40589] regionserver.HeapMemoryManager(220): Stopping 2023-08-18 03:00:03,084 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-18 03:00:03,084 INFO [RS:0;jenkins-hbase4:40589] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-18 03:00:03,085 INFO [RS:0;jenkins-hbase4:40589] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-18 03:00:03,085 INFO [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(3305): Received CLOSE for 1ea064a6a9b903c05e42d398bfe357c7 2023-08-18 03:00:03,085 INFO [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(3305): Received CLOSE for e3c356555f1d3f311cfa4b29c30ff754 2023-08-18 03:00:03,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1ea064a6a9b903c05e42d398bfe357c7, disabling compactions & flushes 2023-08-18 03:00:03,085 INFO [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 03:00:03,085 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. 2023-08-18 03:00:03,085 DEBUG [RS:0;jenkins-hbase4:40589] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1fc2f7f8 to 127.0.0.1:50119 2023-08-18 03:00:03,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. 2023-08-18 03:00:03,085 DEBUG [RS:0;jenkins-hbase4:40589] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 03:00:03,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. after waiting 0 ms 2023-08-18 03:00:03,085 INFO [RS:0;jenkins-hbase4:40589] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-18 03:00:03,086 INFO [RS:0;jenkins-hbase4:40589] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-18 03:00:03,086 INFO [RS:0;jenkins-hbase4:40589] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-18 03:00:03,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. 2023-08-18 03:00:03,086 INFO [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-18 03:00:03,086 INFO [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-08-18 03:00:03,087 DEBUG [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, 1ea064a6a9b903c05e42d398bfe357c7=hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7., e3c356555f1d3f311cfa4b29c30ff754=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754.} 2023-08-18 03:00:03,087 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-18 03:00:03,087 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-18 03:00:03,087 DEBUG [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(1504): Waiting on 1588230740, 1ea064a6a9b903c05e42d398bfe357c7, e3c356555f1d3f311cfa4b29c30ff754 2023-08-18 03:00:03,087 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-18 03:00:03,087 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-18 03:00:03,088 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-18 03:00:03,088 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.10 KB heapSize=5.61 KB 2023-08-18 03:00:03,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/namespace/1ea064a6a9b903c05e42d398bfe357c7/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-18 03:00:03,094 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. 2023-08-18 03:00:03,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1ea064a6a9b903c05e42d398bfe357c7: 2023-08-18 03:00:03,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1692327542022.1ea064a6a9b903c05e42d398bfe357c7. 2023-08-18 03:00:03,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e3c356555f1d3f311cfa4b29c30ff754, disabling compactions & flushes 2023-08-18 03:00:03,095 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 03:00:03,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 03:00:03,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. after waiting 0 ms 2023-08-18 03:00:03,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 03:00:03,095 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing e3c356555f1d3f311cfa4b29c30ff754 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-18 03:00:03,110 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.84 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/.tmp/info/6a7fd1d898e8471b87fba0ae1034c5e3 2023-08-18 03:00:03,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=22 (bloomFilter=true), to=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/.tmp/info/3f82b81fdef54e55904fe26322cf4745 2023-08-18 03:00:03,127 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/.tmp/info/3f82b81fdef54e55904fe26322cf4745 as hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/3f82b81fdef54e55904fe26322cf4745 2023-08-18 03:00:03,136 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=264 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/.tmp/table/0629fd3c8fc94b4fbb8021fa79dd4434 2023-08-18 03:00:03,136 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/3f82b81fdef54e55904fe26322cf4745, entries=1, sequenceid=22, filesize=5.8 K 2023-08-18 03:00:03,138 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for e3c356555f1d3f311cfa4b29c30ff754 in 43ms, sequenceid=22, compaction requested=true 2023-08-18 03:00:03,140 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/e530ce283c064985b1faef78334bb020, hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/fa81d69b1daf4edda044d8cee9009b16, hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/b9c84b180a374c54924745b3312c2b78] to archive 2023-08-18 03:00:03,141 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-18 03:00:03,145 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/e530ce283c064985b1faef78334bb020 to hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/e530ce283c064985b1faef78334bb020 2023-08-18 03:00:03,146 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/fa81d69b1daf4edda044d8cee9009b16 to hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/fa81d69b1daf4edda044d8cee9009b16 2023-08-18 03:00:03,146 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/.tmp/info/6a7fd1d898e8471b87fba0ae1034c5e3 as hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/info/6a7fd1d898e8471b87fba0ae1034c5e3 2023-08-18 03:00:03,149 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/b9c84b180a374c54924745b3312c2b78 to hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/info/b9c84b180a374c54924745b3312c2b78 2023-08-18 03:00:03,155 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/info/6a7fd1d898e8471b87fba0ae1034c5e3, entries=20, sequenceid=14, filesize=7.6 K 2023-08-18 03:00:03,157 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/.tmp/table/0629fd3c8fc94b4fbb8021fa79dd4434 as hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/table/0629fd3c8fc94b4fbb8021fa79dd4434 2023-08-18 03:00:03,163 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/table/0629fd3c8fc94b4fbb8021fa79dd4434, entries=4, sequenceid=14, filesize=4.9 K 2023-08-18 03:00:03,165 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.10 KB/3174, heapSize ~5.33 KB/5456, currentSize=0 B/0 for 1588230740 in 77ms, sequenceid=14, compaction requested=false 2023-08-18 03:00:03,175 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-08-18 03:00:03,175 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-18 03:00:03,176 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-18 03:00:03,176 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-18 03:00:03,176 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-18 03:00:03,182 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e3c356555f1d3f311cfa4b29c30ff754/recovered.edits/25.seqid, newMaxSeqId=25, maxSeqId=1 2023-08-18 03:00:03,183 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 03:00:03,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e3c356555f1d3f311cfa4b29c30ff754: 2023-08-18 03:00:03,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692327542515.e3c356555f1d3f311cfa4b29c30ff754. 2023-08-18 03:00:03,288 INFO [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40589,1692327541373; all regions closed. 2023-08-18 03:00:03,294 DEBUG [RS:0;jenkins-hbase4:40589] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/oldWALs 2023-08-18 03:00:03,294 INFO [RS:0;jenkins-hbase4:40589] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40589%2C1692327541373.meta:.meta(num 1692327541948) 2023-08-18 03:00:03,301 DEBUG [RS:0;jenkins-hbase4:40589] wal.AbstractFSWAL(1028): Moved 2 WAL file(s) to /user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/oldWALs 2023-08-18 03:00:03,301 INFO [RS:0;jenkins-hbase4:40589] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40589%2C1692327541373:(num 1692327603060) 2023-08-18 03:00:03,301 DEBUG [RS:0;jenkins-hbase4:40589] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 03:00:03,301 INFO [RS:0;jenkins-hbase4:40589] regionserver.LeaseManager(133): Closed leases 2023-08-18 03:00:03,301 INFO [RS:0;jenkins-hbase4:40589] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-18 03:00:03,301 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-18 03:00:03,302 INFO [RS:0;jenkins-hbase4:40589] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40589 2023-08-18 03:00:03,304 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40589,1692327541373 2023-08-18 03:00:03,304 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-18 03:00:03,304 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-18 03:00:03,306 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,40589,1692327541373] 2023-08-18 03:00:03,306 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40589,1692327541373; numProcessing=1 2023-08-18 03:00:03,308 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,40589,1692327541373 already deleted, retry=false 2023-08-18 03:00:03,308 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,40589,1692327541373 expired; onlineServers=0 2023-08-18 03:00:03,308 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46191,1692327541325' ***** 2023-08-18 03:00:03,308 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-18 03:00:03,309 DEBUG [M:0;jenkins-hbase4:46191] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@c2486e7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-18 03:00:03,309 INFO [M:0;jenkins-hbase4:46191] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46191,1692327541325 2023-08-18 03:00:03,309 INFO [M:0;jenkins-hbase4:46191] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46191,1692327541325; all regions closed. 2023-08-18 03:00:03,309 DEBUG [M:0;jenkins-hbase4:46191] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 03:00:03,309 DEBUG [M:0;jenkins-hbase4:46191] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-18 03:00:03,309 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-18 03:00:03,309 DEBUG [M:0;jenkins-hbase4:46191] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-18 03:00:03,309 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692327541561] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692327541561,5,FailOnTimeoutGroup] 2023-08-18 03:00:03,309 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692327541561] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692327541561,5,FailOnTimeoutGroup] 2023-08-18 03:00:03,309 INFO [M:0;jenkins-hbase4:46191] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-18 03:00:03,310 INFO [M:0;jenkins-hbase4:46191] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-18 03:00:03,310 INFO [M:0;jenkins-hbase4:46191] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-18 03:00:03,310 DEBUG [M:0;jenkins-hbase4:46191] master.HMaster(1512): Stopping service threads 2023-08-18 03:00:03,310 INFO [M:0;jenkins-hbase4:46191] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-18 03:00:03,310 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-18 03:00:03,311 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:00:03,311 ERROR [M:0;jenkins-hbase4:46191] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-18 03:00:03,311 INFO [M:0;jenkins-hbase4:46191] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-18 03:00:03,311 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-18 03:00:03,311 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-18 03:00:03,311 DEBUG [M:0;jenkins-hbase4:46191] zookeeper.ZKUtil(398): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-18 03:00:03,311 WARN [M:0;jenkins-hbase4:46191] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-18 03:00:03,311 INFO [M:0;jenkins-hbase4:46191] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-18 03:00:03,312 INFO [M:0;jenkins-hbase4:46191] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-18 03:00:03,312 DEBUG [M:0;jenkins-hbase4:46191] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-18 03:00:03,312 INFO [M:0;jenkins-hbase4:46191] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:00:03,312 DEBUG [M:0;jenkins-hbase4:46191] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:00:03,312 DEBUG [M:0;jenkins-hbase4:46191] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-18 03:00:03,312 DEBUG [M:0;jenkins-hbase4:46191] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:00:03,312 INFO [M:0;jenkins-hbase4:46191] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.88 KB heapSize=47.33 KB 2023-08-18 03:00:03,327 INFO [M:0;jenkins-hbase4:46191] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.88 KB at sequenceid=100 (bloomFilter=true), to=hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ff2550ed58e3476ebab4371f4e64391f 2023-08-18 03:00:03,333 INFO [M:0;jenkins-hbase4:46191] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff2550ed58e3476ebab4371f4e64391f 2023-08-18 03:00:03,334 DEBUG [M:0;jenkins-hbase4:46191] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ff2550ed58e3476ebab4371f4e64391f as hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ff2550ed58e3476ebab4371f4e64391f 2023-08-18 03:00:03,340 INFO [M:0;jenkins-hbase4:46191] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff2550ed58e3476ebab4371f4e64391f 2023-08-18 03:00:03,340 INFO [M:0;jenkins-hbase4:46191] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/c543cc0a-e108-37e0-8e68-acb62fcaa13c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ff2550ed58e3476ebab4371f4e64391f, entries=11, sequenceid=100, filesize=6.1 K 2023-08-18 03:00:03,341 INFO [M:0;jenkins-hbase4:46191] regionserver.HRegion(2948): Finished flush of dataSize ~38.88 KB/39812, heapSize ~47.31 KB/48448, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 29ms, sequenceid=100, compaction requested=false 2023-08-18 03:00:03,343 INFO [M:0;jenkins-hbase4:46191] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:00:03,343 DEBUG [M:0;jenkins-hbase4:46191] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-18 03:00:03,346 INFO [M:0;jenkins-hbase4:46191] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-18 03:00:03,346 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-18 03:00:03,347 INFO [M:0;jenkins-hbase4:46191] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46191 2023-08-18 03:00:03,349 DEBUG [M:0;jenkins-hbase4:46191] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,46191,1692327541325 already deleted, retry=false 2023-08-18 03:00:03,406 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 03:00:03,406 INFO [RS:0;jenkins-hbase4:40589] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40589,1692327541373; zookeeper connection closed. 2023-08-18 03:00:03,406 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): regionserver:40589-0x1001d5431de0001, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 03:00:03,407 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2c8fa57d] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2c8fa57d 2023-08-18 03:00:03,407 INFO [Listener at localhost/34121] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-18 03:00:03,506 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 03:00:03,506 INFO [M:0;jenkins-hbase4:46191] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46191,1692327541325; zookeeper connection closed. 2023-08-18 03:00:03,506 DEBUG [Listener at localhost/34121-EventThread] zookeeper.ZKWatcher(600): master:46191-0x1001d5431de0000, quorum=127.0.0.1:50119, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 03:00:03,507 WARN [Listener at localhost/34121] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-18 03:00:03,511 INFO [Listener at localhost/34121] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 03:00:03,615 WARN [BP-716747865-172.31.14.131-1692327540739 heartbeating to localhost/127.0.0.1:38247] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-18 03:00:03,615 WARN [BP-716747865-172.31.14.131-1692327540739 heartbeating to localhost/127.0.0.1:38247] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-716747865-172.31.14.131-1692327540739 (Datanode Uuid b45608b2-049f-44ec-b8c8-8d9fbd2bb658) service to localhost/127.0.0.1:38247 2023-08-18 03:00:03,616 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/cluster_8224e203-f1af-f19e-05b6-812bd3cec471/dfs/data/data3/current/BP-716747865-172.31.14.131-1692327540739] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 03:00:03,616 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/cluster_8224e203-f1af-f19e-05b6-812bd3cec471/dfs/data/data4/current/BP-716747865-172.31.14.131-1692327540739] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 03:00:03,618 WARN [Listener at localhost/34121] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-18 03:00:03,621 INFO [Listener at localhost/34121] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 03:00:03,640 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-18 03:00:03,724 WARN [BP-716747865-172.31.14.131-1692327540739 heartbeating to localhost/127.0.0.1:38247] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-18 03:00:03,724 WARN [BP-716747865-172.31.14.131-1692327540739 heartbeating to localhost/127.0.0.1:38247] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-716747865-172.31.14.131-1692327540739 (Datanode Uuid c8d9ba24-6e75-4f54-82ad-000146c5bb61) service to localhost/127.0.0.1:38247 2023-08-18 03:00:03,725 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/cluster_8224e203-f1af-f19e-05b6-812bd3cec471/dfs/data/data1/current/BP-716747865-172.31.14.131-1692327540739] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 03:00:03,725 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/cluster_8224e203-f1af-f19e-05b6-812bd3cec471/dfs/data/data2/current/BP-716747865-172.31.14.131-1692327540739] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 03:00:03,737 INFO [Listener at localhost/34121] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 03:00:03,852 INFO [Listener at localhost/34121] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-18 03:00:03,868 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-18 03:00:03,878 INFO [Listener at localhost/34121] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=64 (was 78), OpenFileDescriptor=436 (was 467), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=99 (was 119), ProcessCount=175 (was 172) - ProcessCount LEAK? -, AvailableMemoryMB=2242 (was 2337) 2023-08-18 03:00:03,886 INFO [Listener at localhost/34121] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=65, OpenFileDescriptor=436, MaxFileDescriptor=60000, SystemLoadAverage=99, ProcessCount=175, AvailableMemoryMB=2243 2023-08-18 03:00:03,886 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-18 03:00:03,887 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/hadoop.log.dir so I do NOT create it in target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10 2023-08-18 03:00:03,887 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/409d7262-eb17-6969-57a3-a7a34eb2f4aa/hadoop.tmp.dir so I do NOT create it in target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10 2023-08-18 03:00:03,887 INFO [Listener at localhost/34121] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/cluster_dd64d8b1-ed37-efa0-963c-da8e38bf2d1b, deleteOnExit=true 2023-08-18 03:00:03,887 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-18 03:00:03,887 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/test.cache.data in system properties and HBase conf 2023-08-18 03:00:03,887 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/hadoop.tmp.dir in system properties and HBase conf 2023-08-18 03:00:03,887 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/hadoop.log.dir in system properties and HBase conf 2023-08-18 03:00:03,887 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-18 03:00:03,887 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-18 03:00:03,887 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-18 03:00:03,888 DEBUG [Listener at localhost/34121] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-18 03:00:03,888 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-18 03:00:03,888 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-18 03:00:03,888 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-18 03:00:03,888 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-18 03:00:03,888 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-18 03:00:03,888 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-18 03:00:03,888 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-18 03:00:03,889 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-18 03:00:03,889 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-18 03:00:03,889 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/nfs.dump.dir in system properties and HBase conf 2023-08-18 03:00:03,889 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/java.io.tmpdir in system properties and HBase conf 2023-08-18 03:00:03,889 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-18 03:00:03,889 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-18 03:00:03,889 INFO [Listener at localhost/34121] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-18 03:00:03,898 WARN [Listener at localhost/34121] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-18 03:00:03,898 WARN [Listener at localhost/34121] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-18 03:00:03,947 WARN [Listener at localhost/34121] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 03:00:03,950 INFO [Listener at localhost/34121] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 03:00:03,955 INFO [Listener at localhost/34121] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/java.io.tmpdir/Jetty_localhost_37473_hdfs____.ovxezu/webapp 2023-08-18 03:00:04,061 INFO [Listener at localhost/34121] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37473 2023-08-18 03:00:04,117 WARN [Listener at localhost/34121] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-18 03:00:04,117 WARN [Listener at localhost/34121] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-18 03:00:04,182 WARN [Listener at localhost/45289] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 03:00:04,212 WARN [Listener at localhost/45289] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-18 03:00:04,214 WARN [Listener at localhost/45289] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 03:00:04,215 INFO [Listener at localhost/45289] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 03:00:04,220 INFO [Listener at localhost/45289] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/java.io.tmpdir/Jetty_localhost_39759_datanode____3tfg4c/webapp 2023-08-18 03:00:04,331 INFO [Listener at localhost/45289] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39759 2023-08-18 03:00:04,365 WARN [Listener at localhost/43183] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 03:00:04,418 WARN [Listener at localhost/43183] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-18 03:00:04,432 WARN [Listener at localhost/43183] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 03:00:04,434 INFO [Listener at localhost/43183] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 03:00:04,442 INFO [Listener at localhost/43183] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/java.io.tmpdir/Jetty_localhost_35097_datanode____.im11t9/webapp 2023-08-18 03:00:04,564 INFO [Listener at localhost/43183] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35097 2023-08-18 03:00:04,588 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x79120a275ddd4740: Processing first storage report for DS-20db9b42-4f85-449b-8969-e172716f38b9 from datanode caf26e79-78a9-46ea-82cd-efdce0fac523 2023-08-18 03:00:04,589 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x79120a275ddd4740: from storage DS-20db9b42-4f85-449b-8969-e172716f38b9 node DatanodeRegistration(127.0.0.1:34233, datanodeUuid=caf26e79-78a9-46ea-82cd-efdce0fac523, infoPort=33695, infoSecurePort=0, ipcPort=43183, storageInfo=lv=-57;cid=testClusterID;nsid=1039939806;c=1692327603903), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 03:00:04,589 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x79120a275ddd4740: Processing first storage report for DS-363dc0b7-20b0-4689-996a-0bdd155e61a5 from datanode caf26e79-78a9-46ea-82cd-efdce0fac523 2023-08-18 03:00:04,589 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x79120a275ddd4740: from storage DS-363dc0b7-20b0-4689-996a-0bdd155e61a5 node DatanodeRegistration(127.0.0.1:34233, datanodeUuid=caf26e79-78a9-46ea-82cd-efdce0fac523, infoPort=33695, infoSecurePort=0, ipcPort=43183, storageInfo=lv=-57;cid=testClusterID;nsid=1039939806;c=1692327603903), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 03:00:04,593 WARN [Listener at localhost/43431] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 03:00:04,722 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb45f62cefe900c5f: Processing first storage report for DS-8a5f4c93-fc1f-4509-bf62-9f3c09adff4e from datanode 22b65a7e-62c2-431d-b735-4f305df9d260 2023-08-18 03:00:04,723 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb45f62cefe900c5f: from storage DS-8a5f4c93-fc1f-4509-bf62-9f3c09adff4e node DatanodeRegistration(127.0.0.1:33159, datanodeUuid=22b65a7e-62c2-431d-b735-4f305df9d260, infoPort=35705, infoSecurePort=0, ipcPort=43431, storageInfo=lv=-57;cid=testClusterID;nsid=1039939806;c=1692327603903), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 03:00:04,723 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb45f62cefe900c5f: Processing first storage report for DS-269e0a20-7964-42ca-ac06-a2a104c2527c from datanode 22b65a7e-62c2-431d-b735-4f305df9d260 2023-08-18 03:00:04,723 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb45f62cefe900c5f: from storage DS-269e0a20-7964-42ca-ac06-a2a104c2527c node DatanodeRegistration(127.0.0.1:33159, datanodeUuid=22b65a7e-62c2-431d-b735-4f305df9d260, infoPort=35705, infoSecurePort=0, ipcPort=43431, storageInfo=lv=-57;cid=testClusterID;nsid=1039939806;c=1692327603903), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 03:00:04,737 DEBUG [Listener at localhost/43431] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10 2023-08-18 03:00:04,744 INFO [Listener at localhost/43431] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/cluster_dd64d8b1-ed37-efa0-963c-da8e38bf2d1b/zookeeper_0, clientPort=55029, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/cluster_dd64d8b1-ed37-efa0-963c-da8e38bf2d1b/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/cluster_dd64d8b1-ed37-efa0-963c-da8e38bf2d1b/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-18 03:00:04,745 INFO [Listener at localhost/43431] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=55029 2023-08-18 03:00:04,746 INFO [Listener at localhost/43431] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:00:04,747 INFO [Listener at localhost/43431] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:00:04,774 INFO [Listener at localhost/43431] util.FSUtils(471): Created version file at hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3 with version=8 2023-08-18 03:00:04,774 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/hbase-staging 2023-08-18 03:00:04,777 INFO [Listener at localhost/43431] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-18 03:00:04,777 INFO [Listener at localhost/43431] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 03:00:04,777 INFO [Listener at localhost/43431] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-18 03:00:04,777 INFO [Listener at localhost/43431] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-18 03:00:04,777 INFO [Listener at localhost/43431] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 03:00:04,778 INFO [Listener at localhost/43431] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-18 03:00:04,778 INFO [Listener at localhost/43431] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-18 03:00:04,779 INFO [Listener at localhost/43431] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39179 2023-08-18 03:00:04,780 INFO [Listener at localhost/43431] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:00:04,781 INFO [Listener at localhost/43431] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:00:04,783 INFO [Listener at localhost/43431] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39179 connecting to ZooKeeper ensemble=127.0.0.1:55029 2023-08-18 03:00:04,796 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:391790x0, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-18 03:00:04,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:39179-0x1001d5529a90000 connected 2023-08-18 03:00:04,820 DEBUG [Listener at localhost/43431] zookeeper.ZKUtil(164): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-18 03:00:04,820 DEBUG [Listener at localhost/43431] zookeeper.ZKUtil(164): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 03:00:04,821 DEBUG [Listener at localhost/43431] zookeeper.ZKUtil(164): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-18 03:00:04,821 DEBUG [Listener at localhost/43431] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39179 2023-08-18 03:00:04,821 DEBUG [Listener at localhost/43431] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39179 2023-08-18 03:00:04,822 DEBUG [Listener at localhost/43431] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39179 2023-08-18 03:00:04,822 DEBUG [Listener at localhost/43431] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39179 2023-08-18 03:00:04,822 DEBUG [Listener at localhost/43431] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39179 2023-08-18 03:00:04,823 INFO [Listener at localhost/43431] master.HMaster(444): hbase.rootdir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3, hbase.cluster.distributed=false 2023-08-18 03:00:04,836 INFO [Listener at localhost/43431] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-18 03:00:04,836 INFO [Listener at localhost/43431] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 03:00:04,836 INFO [Listener at localhost/43431] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-18 03:00:04,837 INFO [Listener at localhost/43431] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-18 03:00:04,837 INFO [Listener at localhost/43431] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 03:00:04,837 INFO [Listener at localhost/43431] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-18 03:00:04,837 INFO [Listener at localhost/43431] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-18 03:00:04,838 INFO [Listener at localhost/43431] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38805 2023-08-18 03:00:04,838 INFO [Listener at localhost/43431] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-18 03:00:04,839 DEBUG [Listener at localhost/43431] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-18 03:00:04,840 INFO [Listener at localhost/43431] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:00:04,841 INFO [Listener at localhost/43431] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:00:04,842 INFO [Listener at localhost/43431] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38805 connecting to ZooKeeper ensemble=127.0.0.1:55029 2023-08-18 03:00:04,845 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): regionserver:388050x0, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-18 03:00:04,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:38805-0x1001d5529a90001 connected 2023-08-18 03:00:04,846 DEBUG [Listener at localhost/43431] zookeeper.ZKUtil(164): regionserver:38805-0x1001d5529a90001, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-18 03:00:04,847 DEBUG [Listener at localhost/43431] zookeeper.ZKUtil(164): regionserver:38805-0x1001d5529a90001, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 03:00:04,847 DEBUG [Listener at localhost/43431] zookeeper.ZKUtil(164): regionserver:38805-0x1001d5529a90001, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-18 03:00:04,848 DEBUG [Listener at localhost/43431] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38805 2023-08-18 03:00:04,848 DEBUG [Listener at localhost/43431] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38805 2023-08-18 03:00:04,848 DEBUG [Listener at localhost/43431] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38805 2023-08-18 03:00:04,848 DEBUG [Listener at localhost/43431] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38805 2023-08-18 03:00:04,849 DEBUG [Listener at localhost/43431] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38805 2023-08-18 03:00:04,850 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,39179,1692327604776 2023-08-18 03:00:04,851 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-18 03:00:04,851 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,39179,1692327604776 2023-08-18 03:00:04,853 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-18 03:00:04,853 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): regionserver:38805-0x1001d5529a90001, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-18 03:00:04,853 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:00:04,853 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-18 03:00:04,854 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-18 03:00:04,854 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,39179,1692327604776 from backup master directory 2023-08-18 03:00:04,856 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,39179,1692327604776 2023-08-18 03:00:04,856 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-18 03:00:04,856 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-18 03:00:04,856 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,39179,1692327604776 2023-08-18 03:00:04,874 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/hbase.id with ID: f17bcb71-7b3a-4049-9155-84d25f17c2c7 2023-08-18 03:00:04,887 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:00:04,891 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:00:04,900 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x673299c4 to 127.0.0.1:55029 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-18 03:00:04,903 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4de2bedd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-18 03:00:04,903 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-18 03:00:04,904 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-18 03:00:04,905 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-18 03:00:04,906 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/MasterData/data/master/store-tmp 2023-08-18 03:00:04,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:00:04,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-18 03:00:04,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:00:04,926 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:00:04,926 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-18 03:00:04,926 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:00:04,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:00:04,926 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-18 03:00:04,927 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/MasterData/WALs/jenkins-hbase4.apache.org,39179,1692327604776 2023-08-18 03:00:04,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39179%2C1692327604776, suffix=, logDir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/MasterData/WALs/jenkins-hbase4.apache.org,39179,1692327604776, archiveDir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/MasterData/oldWALs, maxLogs=10 2023-08-18 03:00:04,953 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34233,DS-20db9b42-4f85-449b-8969-e172716f38b9,DISK] 2023-08-18 03:00:04,954 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33159,DS-8a5f4c93-fc1f-4509-bf62-9f3c09adff4e,DISK] 2023-08-18 03:00:04,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/MasterData/WALs/jenkins-hbase4.apache.org,39179,1692327604776/jenkins-hbase4.apache.org%2C39179%2C1692327604776.1692327604931 2023-08-18 03:00:04,963 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34233,DS-20db9b42-4f85-449b-8969-e172716f38b9,DISK], DatanodeInfoWithStorage[127.0.0.1:33159,DS-8a5f4c93-fc1f-4509-bf62-9f3c09adff4e,DISK]] 2023-08-18 03:00:04,963 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-18 03:00:04,963 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:00:04,963 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-18 03:00:04,963 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-18 03:00:04,967 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-18 03:00:04,968 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-18 03:00:04,969 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-18 03:00:04,969 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:00:04,970 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-18 03:00:04,971 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-18 03:00:04,976 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-18 03:00:04,979 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-18 03:00:04,979 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=717181, jitterRate=-0.08805795013904572}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-18 03:00:04,979 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-18 03:00:04,981 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-18 03:00:04,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-18 03:00:04,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-18 03:00:04,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-18 03:00:04,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-18 03:00:04,985 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-08-18 03:00:04,985 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-18 03:00:04,989 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-18 03:00:04,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-18 03:00:05,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-18 03:00:05,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-18 03:00:05,003 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-18 03:00:05,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-18 03:00:05,004 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-18 03:00:05,007 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:00:05,008 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-18 03:00:05,009 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-18 03:00:05,010 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-18 03:00:05,012 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-18 03:00:05,012 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:00:05,017 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): regionserver:38805-0x1001d5529a90001, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-18 03:00:05,017 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,39179,1692327604776, sessionid=0x1001d5529a90000, setting cluster-up flag (Was=false) 2023-08-18 03:00:05,020 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:00:05,025 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-18 03:00:05,026 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39179,1692327604776 2023-08-18 03:00:05,030 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:00:05,035 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-18 03:00:05,036 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39179,1692327604776 2023-08-18 03:00:05,038 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/.hbase-snapshot/.tmp 2023-08-18 03:00:05,044 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-18 03:00:05,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 03:00:05,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 03:00:05,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 03:00:05,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 03:00:05,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-18 03:00:05,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:00:05,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-18 03:00:05,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:00:05,059 INFO [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(951): ClusterId : f17bcb71-7b3a-4049-9155-84d25f17c2c7 2023-08-18 03:00:05,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692327635069 2023-08-18 03:00:05,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-18 03:00:05,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-18 03:00:05,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-18 03:00:05,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-18 03:00:05,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-18 03:00:05,070 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-18 03:00:05,073 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-18 03:00:05,074 DEBUG [RS:0;jenkins-hbase4:38805] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-18 03:00:05,075 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-18 03:00:05,075 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-18 03:00:05,075 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-18 03:00:05,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-18 03:00:05,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-18 03:00:05,077 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-18 03:00:05,078 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692327605077,5,FailOnTimeoutGroup] 2023-08-18 03:00:05,078 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692327605078,5,FailOnTimeoutGroup] 2023-08-18 03:00:05,078 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-18 03:00:05,078 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-18 03:00:05,078 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-18 03:00:05,078 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-18 03:00:05,079 DEBUG [RS:0;jenkins-hbase4:38805] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-18 03:00:05,079 DEBUG [RS:0;jenkins-hbase4:38805] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-18 03:00:05,081 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-18 03:00:05,082 DEBUG [RS:0;jenkins-hbase4:38805] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-18 03:00:05,082 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-18 03:00:05,084 DEBUG [RS:0;jenkins-hbase4:38805] zookeeper.ReadOnlyZKClient(139): Connect 0x194ca98e to 127.0.0.1:55029 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-18 03:00:05,101 DEBUG [RS:0;jenkins-hbase4:38805] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@178b54d7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-18 03:00:05,101 DEBUG [RS:0;jenkins-hbase4:38805] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@13b4c2fe, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-18 03:00:05,105 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-18 03:00:05,106 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-18 03:00:05,106 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3 2023-08-18 03:00:05,112 DEBUG [RS:0;jenkins-hbase4:38805] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:38805 2023-08-18 03:00:05,113 INFO [RS:0;jenkins-hbase4:38805] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-18 03:00:05,113 INFO [RS:0;jenkins-hbase4:38805] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-18 03:00:05,113 DEBUG [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(1022): About to register with Master. 2023-08-18 03:00:05,114 INFO [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39179,1692327604776 with isa=jenkins-hbase4.apache.org/172.31.14.131:38805, startcode=1692327604836 2023-08-18 03:00:05,114 DEBUG [RS:0;jenkins-hbase4:38805] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-18 03:00:05,129 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52039, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-08-18 03:00:05,134 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39179] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:05,135 DEBUG [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3 2023-08-18 03:00:05,135 DEBUG [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45289 2023-08-18 03:00:05,135 DEBUG [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-18 03:00:05,136 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-18 03:00:05,137 DEBUG [RS:0;jenkins-hbase4:38805] zookeeper.ZKUtil(162): regionserver:38805-0x1001d5529a90001, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:05,137 WARN [RS:0;jenkins-hbase4:38805] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-18 03:00:05,137 INFO [RS:0;jenkins-hbase4:38805] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-18 03:00:05,137 DEBUG [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/WALs/jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:05,167 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38805,1692327604836] 2023-08-18 03:00:05,168 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:00:05,169 DEBUG [RS:0;jenkins-hbase4:38805] zookeeper.ZKUtil(162): regionserver:38805-0x1001d5529a90001, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:05,170 DEBUG [RS:0;jenkins-hbase4:38805] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-18 03:00:05,170 INFO [RS:0;jenkins-hbase4:38805] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-18 03:00:05,173 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-18 03:00:05,176 INFO [RS:0;jenkins-hbase4:38805] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-18 03:00:05,177 INFO [RS:0;jenkins-hbase4:38805] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-18 03:00:05,177 INFO [RS:0;jenkins-hbase4:38805] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-18 03:00:05,178 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/info 2023-08-18 03:00:05,179 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-18 03:00:05,179 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:00:05,180 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-18 03:00:05,184 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/rep_barrier 2023-08-18 03:00:05,184 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-18 03:00:05,185 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:00:05,185 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-18 03:00:05,189 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/table 2023-08-18 03:00:05,190 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-18 03:00:05,191 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:00:05,189 INFO [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-18 03:00:05,193 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740 2023-08-18 03:00:05,194 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740 2023-08-18 03:00:05,194 INFO [RS:0;jenkins-hbase4:38805] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-18 03:00:05,195 DEBUG [RS:0;jenkins-hbase4:38805] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:00:05,195 DEBUG [RS:0;jenkins-hbase4:38805] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:00:05,195 DEBUG [RS:0;jenkins-hbase4:38805] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:00:05,195 DEBUG [RS:0;jenkins-hbase4:38805] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:00:05,195 DEBUG [RS:0;jenkins-hbase4:38805] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:00:05,195 DEBUG [RS:0;jenkins-hbase4:38805] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-18 03:00:05,195 DEBUG [RS:0;jenkins-hbase4:38805] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:00:05,195 DEBUG [RS:0;jenkins-hbase4:38805] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:00:05,195 DEBUG [RS:0;jenkins-hbase4:38805] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:00:05,195 DEBUG [RS:0;jenkins-hbase4:38805] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:00:05,196 INFO [RS:0;jenkins-hbase4:38805] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-18 03:00:05,196 INFO [RS:0;jenkins-hbase4:38805] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-18 03:00:05,197 INFO [RS:0;jenkins-hbase4:38805] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-18 03:00:05,198 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-18 03:00:05,200 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-18 03:00:05,220 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-18 03:00:05,221 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=724783, jitterRate=-0.07839182019233704}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-18 03:00:05,221 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-18 03:00:05,221 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-18 03:00:05,221 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-18 03:00:05,221 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-18 03:00:05,221 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-18 03:00:05,221 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-18 03:00:05,223 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-18 03:00:05,223 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-18 03:00:05,225 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-18 03:00:05,225 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-18 03:00:05,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-18 03:00:05,226 INFO [RS:0;jenkins-hbase4:38805] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-18 03:00:05,226 INFO [RS:0;jenkins-hbase4:38805] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38805,1692327604836-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-18 03:00:05,230 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-18 03:00:05,233 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-18 03:00:05,253 INFO [RS:0;jenkins-hbase4:38805] regionserver.Replication(203): jenkins-hbase4.apache.org,38805,1692327604836 started 2023-08-18 03:00:05,253 INFO [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,38805,1692327604836, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38805, sessionid=0x1001d5529a90001 2023-08-18 03:00:05,253 DEBUG [RS:0;jenkins-hbase4:38805] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-18 03:00:05,254 DEBUG [RS:0;jenkins-hbase4:38805] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:05,254 DEBUG [RS:0;jenkins-hbase4:38805] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38805,1692327604836' 2023-08-18 03:00:05,254 DEBUG [RS:0;jenkins-hbase4:38805] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-18 03:00:05,254 DEBUG [RS:0;jenkins-hbase4:38805] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-18 03:00:05,255 DEBUG [RS:0;jenkins-hbase4:38805] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-18 03:00:05,255 DEBUG [RS:0;jenkins-hbase4:38805] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-18 03:00:05,255 DEBUG [RS:0;jenkins-hbase4:38805] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:05,255 DEBUG [RS:0;jenkins-hbase4:38805] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38805,1692327604836' 2023-08-18 03:00:05,255 DEBUG [RS:0;jenkins-hbase4:38805] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-18 03:00:05,255 DEBUG [RS:0;jenkins-hbase4:38805] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-18 03:00:05,256 DEBUG [RS:0;jenkins-hbase4:38805] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-18 03:00:05,256 INFO [RS:0;jenkins-hbase4:38805] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-18 03:00:05,256 INFO [RS:0;jenkins-hbase4:38805] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-18 03:00:05,358 INFO [RS:0;jenkins-hbase4:38805] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38805%2C1692327604836, suffix=, logDir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/WALs/jenkins-hbase4.apache.org,38805,1692327604836, archiveDir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/oldWALs, maxLogs=32 2023-08-18 03:00:05,375 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34233,DS-20db9b42-4f85-449b-8969-e172716f38b9,DISK] 2023-08-18 03:00:05,375 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33159,DS-8a5f4c93-fc1f-4509-bf62-9f3c09adff4e,DISK] 2023-08-18 03:00:05,377 INFO [RS:0;jenkins-hbase4:38805] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/WALs/jenkins-hbase4.apache.org,38805,1692327604836/jenkins-hbase4.apache.org%2C38805%2C1692327604836.1692327605359 2023-08-18 03:00:05,378 DEBUG [RS:0;jenkins-hbase4:38805] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34233,DS-20db9b42-4f85-449b-8969-e172716f38b9,DISK], DatanodeInfoWithStorage[127.0.0.1:33159,DS-8a5f4c93-fc1f-4509-bf62-9f3c09adff4e,DISK]] 2023-08-18 03:00:05,384 DEBUG [jenkins-hbase4:39179] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-18 03:00:05,384 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38805,1692327604836, state=OPENING 2023-08-18 03:00:05,386 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-18 03:00:05,389 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:00:05,389 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38805,1692327604836}] 2023-08-18 03:00:05,389 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-18 03:00:05,543 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:05,544 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-18 03:00:05,546 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38964, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-18 03:00:05,553 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-18 03:00:05,553 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-18 03:00:05,565 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38805%2C1692327604836.meta, suffix=.meta, logDir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/WALs/jenkins-hbase4.apache.org,38805,1692327604836, archiveDir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/oldWALs, maxLogs=32 2023-08-18 03:00:05,588 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34233,DS-20db9b42-4f85-449b-8969-e172716f38b9,DISK] 2023-08-18 03:00:05,588 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33159,DS-8a5f4c93-fc1f-4509-bf62-9f3c09adff4e,DISK] 2023-08-18 03:00:05,597 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/WALs/jenkins-hbase4.apache.org,38805,1692327604836/jenkins-hbase4.apache.org%2C38805%2C1692327604836.meta.1692327605566.meta 2023-08-18 03:00:05,597 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34233,DS-20db9b42-4f85-449b-8969-e172716f38b9,DISK], DatanodeInfoWithStorage[127.0.0.1:33159,DS-8a5f4c93-fc1f-4509-bf62-9f3c09adff4e,DISK]] 2023-08-18 03:00:05,597 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-18 03:00:05,598 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-18 03:00:05,598 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-18 03:00:05,598 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-18 03:00:05,598 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-18 03:00:05,598 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:00:05,598 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-18 03:00:05,598 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-18 03:00:05,600 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-18 03:00:05,601 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/info 2023-08-18 03:00:05,601 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/info 2023-08-18 03:00:05,602 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-18 03:00:05,602 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:00:05,603 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-18 03:00:05,603 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/rep_barrier 2023-08-18 03:00:05,604 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/rep_barrier 2023-08-18 03:00:05,604 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-18 03:00:05,605 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:00:05,605 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-18 03:00:05,606 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/table 2023-08-18 03:00:05,606 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/table 2023-08-18 03:00:05,607 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-18 03:00:05,607 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:00:05,608 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740 2023-08-18 03:00:05,610 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740 2023-08-18 03:00:05,612 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-18 03:00:05,614 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-18 03:00:05,615 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=799967, jitterRate=0.017211496829986572}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-18 03:00:05,615 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-18 03:00:05,616 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1692327605543 2023-08-18 03:00:05,619 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-18 03:00:05,620 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-18 03:00:05,620 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38805,1692327604836, state=OPEN 2023-08-18 03:00:05,623 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-18 03:00:05,623 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-18 03:00:05,626 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-18 03:00:05,626 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38805,1692327604836 in 234 msec 2023-08-18 03:00:05,628 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-18 03:00:05,628 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 401 msec 2023-08-18 03:00:05,630 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 585 msec 2023-08-18 03:00:05,630 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1692327605630, completionTime=-1 2023-08-18 03:00:05,630 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-18 03:00:05,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-18 03:00:05,632 DEBUG [hconnection-0x67e42226-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-18 03:00:05,634 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38966, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-18 03:00:05,636 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-18 03:00:05,636 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692327665636 2023-08-18 03:00:05,636 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692327725636 2023-08-18 03:00:05,636 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-18 03:00:05,642 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39179,1692327604776-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-18 03:00:05,642 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39179,1692327604776-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-18 03:00:05,642 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39179,1692327604776-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-18 03:00:05,642 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:39179, period=300000, unit=MILLISECONDS is enabled. 2023-08-18 03:00:05,642 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-18 03:00:05,642 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-18 03:00:05,642 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-18 03:00:05,643 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-18 03:00:05,643 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-18 03:00:05,645 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-18 03:00:05,646 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-18 03:00:05,647 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/.tmp/data/hbase/namespace/6b5011f83909e605795a74cdf612ac58 2023-08-18 03:00:05,648 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/.tmp/data/hbase/namespace/6b5011f83909e605795a74cdf612ac58 empty. 2023-08-18 03:00:05,648 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/.tmp/data/hbase/namespace/6b5011f83909e605795a74cdf612ac58 2023-08-18 03:00:05,648 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-18 03:00:05,661 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-18 03:00:05,662 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6b5011f83909e605795a74cdf612ac58, NAME => 'hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/.tmp 2023-08-18 03:00:05,671 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:00:05,671 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 6b5011f83909e605795a74cdf612ac58, disabling compactions & flushes 2023-08-18 03:00:05,671 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58. 2023-08-18 03:00:05,671 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58. 2023-08-18 03:00:05,671 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58. after waiting 0 ms 2023-08-18 03:00:05,671 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58. 2023-08-18 03:00:05,671 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58. 2023-08-18 03:00:05,671 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 6b5011f83909e605795a74cdf612ac58: 2023-08-18 03:00:05,674 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-18 03:00:05,675 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692327605675"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692327605675"}]},"ts":"1692327605675"} 2023-08-18 03:00:05,677 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-18 03:00:05,678 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-18 03:00:05,678 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692327605678"}]},"ts":"1692327605678"} 2023-08-18 03:00:05,680 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-18 03:00:05,687 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6b5011f83909e605795a74cdf612ac58, ASSIGN}] 2023-08-18 03:00:05,689 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6b5011f83909e605795a74cdf612ac58, ASSIGN 2023-08-18 03:00:05,690 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=6b5011f83909e605795a74cdf612ac58, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38805,1692327604836; forceNewPlan=false, retain=false 2023-08-18 03:00:05,841 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=6b5011f83909e605795a74cdf612ac58, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:05,841 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692327605841"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692327605841"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692327605841"}]},"ts":"1692327605841"} 2023-08-18 03:00:05,843 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 6b5011f83909e605795a74cdf612ac58, server=jenkins-hbase4.apache.org,38805,1692327604836}] 2023-08-18 03:00:06,000 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58. 2023-08-18 03:00:06,000 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6b5011f83909e605795a74cdf612ac58, NAME => 'hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58.', STARTKEY => '', ENDKEY => ''} 2023-08-18 03:00:06,000 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 6b5011f83909e605795a74cdf612ac58 2023-08-18 03:00:06,000 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:00:06,001 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6b5011f83909e605795a74cdf612ac58 2023-08-18 03:00:06,001 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6b5011f83909e605795a74cdf612ac58 2023-08-18 03:00:06,002 INFO [StoreOpener-6b5011f83909e605795a74cdf612ac58-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 6b5011f83909e605795a74cdf612ac58 2023-08-18 03:00:06,003 DEBUG [StoreOpener-6b5011f83909e605795a74cdf612ac58-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/namespace/6b5011f83909e605795a74cdf612ac58/info 2023-08-18 03:00:06,003 DEBUG [StoreOpener-6b5011f83909e605795a74cdf612ac58-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/namespace/6b5011f83909e605795a74cdf612ac58/info 2023-08-18 03:00:06,004 INFO [StoreOpener-6b5011f83909e605795a74cdf612ac58-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b5011f83909e605795a74cdf612ac58 columnFamilyName info 2023-08-18 03:00:06,004 INFO [StoreOpener-6b5011f83909e605795a74cdf612ac58-1] regionserver.HStore(310): Store=6b5011f83909e605795a74cdf612ac58/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:00:06,005 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/namespace/6b5011f83909e605795a74cdf612ac58 2023-08-18 03:00:06,006 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/namespace/6b5011f83909e605795a74cdf612ac58 2023-08-18 03:00:06,009 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6b5011f83909e605795a74cdf612ac58 2023-08-18 03:00:06,011 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/namespace/6b5011f83909e605795a74cdf612ac58/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-18 03:00:06,012 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6b5011f83909e605795a74cdf612ac58; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=862930, jitterRate=0.09727251529693604}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-18 03:00:06,012 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6b5011f83909e605795a74cdf612ac58: 2023-08-18 03:00:06,013 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58., pid=6, masterSystemTime=1692327605996 2023-08-18 03:00:06,015 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58. 2023-08-18 03:00:06,015 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58. 2023-08-18 03:00:06,016 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=6b5011f83909e605795a74cdf612ac58, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:06,016 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692327606016"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692327606016"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692327606016"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692327606016"}]},"ts":"1692327606016"} 2023-08-18 03:00:06,020 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-18 03:00:06,020 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 6b5011f83909e605795a74cdf612ac58, server=jenkins-hbase4.apache.org,38805,1692327604836 in 175 msec 2023-08-18 03:00:06,022 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-18 03:00:06,023 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=6b5011f83909e605795a74cdf612ac58, ASSIGN in 333 msec 2023-08-18 03:00:06,023 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-18 03:00:06,024 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692327606024"}]},"ts":"1692327606024"} 2023-08-18 03:00:06,025 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-18 03:00:06,027 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-18 03:00:06,029 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 385 msec 2023-08-18 03:00:06,044 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-18 03:00:06,046 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-18 03:00:06,046 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:00:06,050 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-18 03:00:06,071 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-18 03:00:06,079 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 28 msec 2023-08-18 03:00:06,083 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-18 03:00:06,096 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-18 03:00:06,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 17 msec 2023-08-18 03:00:06,119 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-18 03:00:06,127 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-18 03:00:06,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.271sec 2023-08-18 03:00:06,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-18 03:00:06,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-18 03:00:06,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-18 03:00:06,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39179,1692327604776-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-18 03:00:06,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39179,1692327604776-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-18 03:00:06,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-18 03:00:06,157 DEBUG [Listener at localhost/43431] zookeeper.ReadOnlyZKClient(139): Connect 0x2630ef71 to 127.0.0.1:55029 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-18 03:00:06,162 DEBUG [Listener at localhost/43431] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@14abad4e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-18 03:00:06,163 DEBUG [hconnection-0x4732fc7-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-18 03:00:06,165 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38982, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-18 03:00:06,166 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,39179,1692327604776 2023-08-18 03:00:06,167 INFO [Listener at localhost/43431] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:00:06,170 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-18 03:00:06,170 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:00:06,170 INFO [Listener at localhost/43431] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-18 03:00:06,172 DEBUG [Listener at localhost/43431] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-18 03:00:06,175 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47498, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-18 03:00:06,176 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39179] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-08-18 03:00:06,176 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39179] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-08-18 03:00:06,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39179] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-18 03:00:06,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39179] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRolling 2023-08-18 03:00:06,181 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-08-18 03:00:06,181 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39179] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRolling" procId is: 9 2023-08-18 03:00:06,182 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-18 03:00:06,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39179] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-18 03:00:06,186 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/.tmp/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:06,187 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/.tmp/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a empty. 2023-08-18 03:00:06,187 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/.tmp/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:06,187 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRolling regions 2023-08-18 03:00:06,205 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/.tmp/data/default/TestLogRolling-testLogRolling/.tabledesc/.tableinfo.0000000001 2023-08-18 03:00:06,206 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => a65569a4fb904c2acd072fd335815f9a, NAME => 'TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/.tmp 2023-08-18 03:00:06,217 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:00:06,217 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1604): Closing a65569a4fb904c2acd072fd335815f9a, disabling compactions & flushes 2023-08-18 03:00:06,217 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. 2023-08-18 03:00:06,217 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. 2023-08-18 03:00:06,217 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. after waiting 0 ms 2023-08-18 03:00:06,217 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. 2023-08-18 03:00:06,217 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. 2023-08-18 03:00:06,217 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1558): Region close journal for a65569a4fb904c2acd072fd335815f9a: 2023-08-18 03:00:06,220 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ADD_TO_META 2023-08-18 03:00:06,222 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1692327606222"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692327606222"}]},"ts":"1692327606222"} 2023-08-18 03:00:06,224 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-18 03:00:06,225 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-18 03:00:06,225 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692327606225"}]},"ts":"1692327606225"} 2023-08-18 03:00:06,227 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLING in hbase:meta 2023-08-18 03:00:06,231 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=a65569a4fb904c2acd072fd335815f9a, ASSIGN}] 2023-08-18 03:00:06,233 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=a65569a4fb904c2acd072fd335815f9a, ASSIGN 2023-08-18 03:00:06,234 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=a65569a4fb904c2acd072fd335815f9a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38805,1692327604836; forceNewPlan=false, retain=false 2023-08-18 03:00:06,385 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=a65569a4fb904c2acd072fd335815f9a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:06,386 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1692327606385"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692327606385"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692327606385"}]},"ts":"1692327606385"} 2023-08-18 03:00:06,388 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure a65569a4fb904c2acd072fd335815f9a, server=jenkins-hbase4.apache.org,38805,1692327604836}] 2023-08-18 03:00:06,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. 2023-08-18 03:00:06,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a65569a4fb904c2acd072fd335815f9a, NAME => 'TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.', STARTKEY => '', ENDKEY => ''} 2023-08-18 03:00:06,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:06,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:00:06,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:06,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:06,547 INFO [StoreOpener-a65569a4fb904c2acd072fd335815f9a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:06,548 DEBUG [StoreOpener-a65569a4fb904c2acd072fd335815f9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info 2023-08-18 03:00:06,548 DEBUG [StoreOpener-a65569a4fb904c2acd072fd335815f9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info 2023-08-18 03:00:06,548 INFO [StoreOpener-a65569a4fb904c2acd072fd335815f9a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a65569a4fb904c2acd072fd335815f9a columnFamilyName info 2023-08-18 03:00:06,549 INFO [StoreOpener-a65569a4fb904c2acd072fd335815f9a-1] regionserver.HStore(310): Store=a65569a4fb904c2acd072fd335815f9a/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:00:06,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:06,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:06,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:06,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-18 03:00:06,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a65569a4fb904c2acd072fd335815f9a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=850950, jitterRate=0.0820399820804596}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-18 03:00:06,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a65569a4fb904c2acd072fd335815f9a: 2023-08-18 03:00:06,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a., pid=11, masterSystemTime=1692327606540 2023-08-18 03:00:06,587 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=a65569a4fb904c2acd072fd335815f9a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:06,587 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1692327606587"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692327606587"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692327606587"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692327606587"}]},"ts":"1692327606587"} 2023-08-18 03:00:06,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-18 03:00:06,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure a65569a4fb904c2acd072fd335815f9a, server=jenkins-hbase4.apache.org,38805,1692327604836 in 201 msec 2023-08-18 03:00:06,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. 2023-08-18 03:00:06,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. 2023-08-18 03:00:06,594 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-18 03:00:06,594 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=a65569a4fb904c2acd072fd335815f9a, ASSIGN in 360 msec 2023-08-18 03:00:06,595 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-18 03:00:06,595 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692327606595"}]},"ts":"1692327606595"} 2023-08-18 03:00:06,597 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLED in hbase:meta 2023-08-18 03:00:06,599 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_POST_OPERATION 2023-08-18 03:00:06,601 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRolling in 422 msec 2023-08-18 03:00:08,624 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-18 03:00:11,171 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-18 03:00:11,171 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-18 03:00:11,171 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRolling' 2023-08-18 03:00:16,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39179] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-18 03:00:16,184 INFO [Listener at localhost/43431] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRolling, procId: 9 completed 2023-08-18 03:00:16,187 DEBUG [Listener at localhost/43431] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRolling 2023-08-18 03:00:16,187 DEBUG [Listener at localhost/43431] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. 2023-08-18 03:00:16,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:16,205 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing a65569a4fb904c2acd072fd335815f9a 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-18 03:00:16,219 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=12 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp/info/94d6b771786242df8bafe9743217af59 2023-08-18 03:00:16,227 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp/info/94d6b771786242df8bafe9743217af59 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/94d6b771786242df8bafe9743217af59 2023-08-18 03:00:16,233 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=a65569a4fb904c2acd072fd335815f9a, server=jenkins-hbase4.apache.org,38805,1692327604836 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-18 03:00:16,234 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/94d6b771786242df8bafe9743217af59, entries=8, sequenceid=12, filesize=13.0 K 2023-08-18 03:00:16,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] ipc.CallRunner(144): callId: 38 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:38982 deadline: 1692327626233, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=a65569a4fb904c2acd072fd335815f9a, server=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:16,235 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for a65569a4fb904c2acd072fd335815f9a in 30ms, sequenceid=12, compaction requested=false 2023-08-18 03:00:16,236 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for a65569a4fb904c2acd072fd335815f9a: 2023-08-18 03:00:26,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:26,328 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing a65569a4fb904c2acd072fd335815f9a 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-18 03:00:26,344 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=37 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp/info/4b500232ed4042edb8338847486e97b8 2023-08-18 03:00:26,350 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp/info/4b500232ed4042edb8338847486e97b8 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4b500232ed4042edb8338847486e97b8 2023-08-18 03:00:26,357 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4b500232ed4042edb8338847486e97b8, entries=22, sequenceid=37, filesize=27.4 K 2023-08-18 03:00:26,358 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=2.05 KB/2102 for a65569a4fb904c2acd072fd335815f9a in 30ms, sequenceid=37, compaction requested=false 2023-08-18 03:00:26,358 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for a65569a4fb904c2acd072fd335815f9a: 2023-08-18 03:00:26,358 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=40.3 K, sizeToCheck=16.0 K 2023-08-18 03:00:26,358 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-18 03:00:26,359 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4b500232ed4042edb8338847486e97b8 because midkey is the same as first or last row 2023-08-18 03:00:28,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:28,341 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing a65569a4fb904c2acd072fd335815f9a 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-18 03:00:28,374 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=48 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp/info/6889c9b9c4d94cb087ede86c452b05bc 2023-08-18 03:00:28,379 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=a65569a4fb904c2acd072fd335815f9a, server=jenkins-hbase4.apache.org,38805,1692327604836 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-18 03:00:28,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] ipc.CallRunner(144): callId: 70 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:38982 deadline: 1692327638379, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=a65569a4fb904c2acd072fd335815f9a, server=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:28,383 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp/info/6889c9b9c4d94cb087ede86c452b05bc as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/6889c9b9c4d94cb087ede86c452b05bc 2023-08-18 03:00:28,389 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/6889c9b9c4d94cb087ede86c452b05bc, entries=8, sequenceid=48, filesize=13.0 K 2023-08-18 03:00:28,391 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for a65569a4fb904c2acd072fd335815f9a in 49ms, sequenceid=48, compaction requested=true 2023-08-18 03:00:28,391 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for a65569a4fb904c2acd072fd335815f9a: 2023-08-18 03:00:28,391 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=53.3 K, sizeToCheck=16.0 K 2023-08-18 03:00:28,391 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-18 03:00:28,391 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4b500232ed4042edb8338847486e97b8 because midkey is the same as first or last row 2023-08-18 03:00:28,393 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:00:28,393 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-18 03:00:28,395 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 54576 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-18 03:00:28,395 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1912): a65569a4fb904c2acd072fd335815f9a/info is initiating minor compaction (all files) 2023-08-18 03:00:28,395 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a65569a4fb904c2acd072fd335815f9a/info in TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. 2023-08-18 03:00:28,396 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/94d6b771786242df8bafe9743217af59, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4b500232ed4042edb8338847486e97b8, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/6889c9b9c4d94cb087ede86c452b05bc] into tmpdir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp, totalSize=53.3 K 2023-08-18 03:00:28,396 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting 94d6b771786242df8bafe9743217af59, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1692327616191 2023-08-18 03:00:28,397 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting 4b500232ed4042edb8338847486e97b8, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=37, earliestPutTs=1692327616206 2023-08-18 03:00:28,397 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting 6889c9b9c4d94cb087ede86c452b05bc, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1692327626329 2023-08-18 03:00:28,418 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] throttle.PressureAwareThroughputController(145): a65569a4fb904c2acd072fd335815f9a#info#compaction#18 average throughput is 19.04 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-18 03:00:28,438 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp/info/ae01b10a838347d89b20599a239aa5bb as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/ae01b10a838347d89b20599a239aa5bb 2023-08-18 03:00:28,444 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in a65569a4fb904c2acd072fd335815f9a/info of a65569a4fb904c2acd072fd335815f9a into ae01b10a838347d89b20599a239aa5bb(size=43.9 K), total size for store is 43.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-18 03:00:28,444 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a65569a4fb904c2acd072fd335815f9a: 2023-08-18 03:00:28,445 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a., storeName=a65569a4fb904c2acd072fd335815f9a/info, priority=13, startTime=1692327628393; duration=0sec 2023-08-18 03:00:28,446 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=43.9 K, sizeToCheck=16.0 K 2023-08-18 03:00:28,446 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-18 03:00:28,446 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.StoreUtils(129): cannot split hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/ae01b10a838347d89b20599a239aa5bb because midkey is the same as first or last row 2023-08-18 03:00:28,446 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:00:38,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:38,412 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing a65569a4fb904c2acd072fd335815f9a 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-18 03:00:38,436 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=74 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp/info/0a837b468fef4dbba4426dc5ebb78ad6 2023-08-18 03:00:38,451 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp/info/0a837b468fef4dbba4426dc5ebb78ad6 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/0a837b468fef4dbba4426dc5ebb78ad6 2023-08-18 03:00:38,458 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/0a837b468fef4dbba4426dc5ebb78ad6, entries=22, sequenceid=74, filesize=27.4 K 2023-08-18 03:00:38,459 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=4.11 KB/4204 for a65569a4fb904c2acd072fd335815f9a in 47ms, sequenceid=74, compaction requested=false 2023-08-18 03:00:38,459 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for a65569a4fb904c2acd072fd335815f9a: 2023-08-18 03:00:38,459 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=71.3 K, sizeToCheck=16.0 K 2023-08-18 03:00:38,459 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-18 03:00:38,459 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/ae01b10a838347d89b20599a239aa5bb because midkey is the same as first or last row 2023-08-18 03:00:40,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:40,426 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing a65569a4fb904c2acd072fd335815f9a 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-18 03:00:40,440 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=85 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp/info/a5f44d8b95e740a287a5f4b8fac625a6 2023-08-18 03:00:40,447 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp/info/a5f44d8b95e740a287a5f4b8fac625a6 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/a5f44d8b95e740a287a5f4b8fac625a6 2023-08-18 03:00:40,453 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=a65569a4fb904c2acd072fd335815f9a, server=jenkins-hbase4.apache.org,38805,1692327604836 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-18 03:00:40,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] ipc.CallRunner(144): callId: 102 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:38982 deadline: 1692327650453, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=a65569a4fb904c2acd072fd335815f9a, server=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:40,453 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/a5f44d8b95e740a287a5f4b8fac625a6, entries=8, sequenceid=85, filesize=13.0 K 2023-08-18 03:00:40,454 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for a65569a4fb904c2acd072fd335815f9a in 28ms, sequenceid=85, compaction requested=true 2023-08-18 03:00:40,454 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for a65569a4fb904c2acd072fd335815f9a: 2023-08-18 03:00:40,454 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=84.3 K, sizeToCheck=16.0 K 2023-08-18 03:00:40,454 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-18 03:00:40,454 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/ae01b10a838347d89b20599a239aa5bb because midkey is the same as first or last row 2023-08-18 03:00:40,454 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-18 03:00:40,454 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-18 03:00:40,456 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 86295 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-18 03:00:40,456 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HStore(1912): a65569a4fb904c2acd072fd335815f9a/info is initiating minor compaction (all files) 2023-08-18 03:00:40,456 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HRegion(2259): Starting compaction of a65569a4fb904c2acd072fd335815f9a/info in TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. 2023-08-18 03:00:40,456 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/ae01b10a838347d89b20599a239aa5bb, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/0a837b468fef4dbba4426dc5ebb78ad6, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/a5f44d8b95e740a287a5f4b8fac625a6] into tmpdir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp, totalSize=84.3 K 2023-08-18 03:00:40,456 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] compactions.Compactor(207): Compacting ae01b10a838347d89b20599a239aa5bb, keycount=38, bloomtype=ROW, size=43.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1692327616191 2023-08-18 03:00:40,457 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] compactions.Compactor(207): Compacting 0a837b468fef4dbba4426dc5ebb78ad6, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=74, earliestPutTs=1692327628342 2023-08-18 03:00:40,457 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] compactions.Compactor(207): Compacting a5f44d8b95e740a287a5f4b8fac625a6, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1692327638416 2023-08-18 03:00:40,469 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] throttle.PressureAwareThroughputController(145): a65569a4fb904c2acd072fd335815f9a#info#compaction#21 average throughput is 22.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-18 03:00:40,486 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp/info/4eefa1f4973a48f89640518373c5e383 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4eefa1f4973a48f89640518373c5e383 2023-08-18 03:00:40,493 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in a65569a4fb904c2acd072fd335815f9a/info of a65569a4fb904c2acd072fd335815f9a into 4eefa1f4973a48f89640518373c5e383(size=75.0 K), total size for store is 75.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-18 03:00:40,493 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for a65569a4fb904c2acd072fd335815f9a: 2023-08-18 03:00:40,493 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a., storeName=a65569a4fb904c2acd072fd335815f9a/info, priority=13, startTime=1692327640454; duration=0sec 2023-08-18 03:00:40,493 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=75.0 K, sizeToCheck=16.0 K 2023-08-18 03:00:40,493 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-18 03:00:40,494 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.CompactSplit(227): Splitting TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a., compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:00:40,495 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:00:40,496 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39179] assignment.AssignmentManager(1140): Split request from jenkins-hbase4.apache.org,38805,1692327604836, parent={ENCODED => a65569a4fb904c2acd072fd335815f9a, NAME => 'TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.', STARTKEY => '', ENDKEY => ''} splitKey=row0064 2023-08-18 03:00:40,504 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39179] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:40,510 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39179] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=a65569a4fb904c2acd072fd335815f9a, daughterA=a9284c12aea305493291bf3e413c6c3d, daughterB=1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:00:40,511 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=a65569a4fb904c2acd072fd335815f9a, daughterA=a9284c12aea305493291bf3e413c6c3d, daughterB=1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:00:40,511 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=a65569a4fb904c2acd072fd335815f9a, daughterA=a9284c12aea305493291bf3e413c6c3d, daughterB=1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:00:40,511 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=a65569a4fb904c2acd072fd335815f9a, daughterA=a9284c12aea305493291bf3e413c6c3d, daughterB=1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:00:40,518 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=a65569a4fb904c2acd072fd335815f9a, UNASSIGN}] 2023-08-18 03:00:40,520 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=a65569a4fb904c2acd072fd335815f9a, UNASSIGN 2023-08-18 03:00:40,520 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=a65569a4fb904c2acd072fd335815f9a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:40,521 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1692327640520"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692327640520"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692327640520"}]},"ts":"1692327640520"} 2023-08-18 03:00:40,522 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure a65569a4fb904c2acd072fd335815f9a, server=jenkins-hbase4.apache.org,38805,1692327604836}] 2023-08-18 03:00:40,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:40,680 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a65569a4fb904c2acd072fd335815f9a, disabling compactions & flushes 2023-08-18 03:00:40,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. 2023-08-18 03:00:40,681 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. 2023-08-18 03:00:40,681 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. after waiting 0 ms 2023-08-18 03:00:40,681 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. 2023-08-18 03:00:40,681 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing a65569a4fb904c2acd072fd335815f9a 1/1 column families, dataSize=21.55 KB heapSize=23.38 KB 2023-08-18 03:00:40,692 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=21.55 KB at sequenceid=110 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp/info/ca65f4e0a1394a7997b39f3079177db1 2023-08-18 03:00:40,697 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.tmp/info/ca65f4e0a1394a7997b39f3079177db1 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/ca65f4e0a1394a7997b39f3079177db1 2023-08-18 03:00:40,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/ca65f4e0a1394a7997b39f3079177db1, entries=21, sequenceid=110, filesize=26.3 K 2023-08-18 03:00:40,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~21.55 KB/22071, heapSize ~23.37 KB/23928, currentSize=0 B/0 for a65569a4fb904c2acd072fd335815f9a in 23ms, sequenceid=110, compaction requested=false 2023-08-18 03:00:40,710 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/94d6b771786242df8bafe9743217af59, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4b500232ed4042edb8338847486e97b8, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/ae01b10a838347d89b20599a239aa5bb, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/6889c9b9c4d94cb087ede86c452b05bc, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/0a837b468fef4dbba4426dc5ebb78ad6, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/a5f44d8b95e740a287a5f4b8fac625a6] to archive 2023-08-18 03:00:40,711 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-18 03:00:40,713 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/94d6b771786242df8bafe9743217af59 to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/94d6b771786242df8bafe9743217af59 2023-08-18 03:00:40,715 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4b500232ed4042edb8338847486e97b8 to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4b500232ed4042edb8338847486e97b8 2023-08-18 03:00:40,716 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/ae01b10a838347d89b20599a239aa5bb to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/ae01b10a838347d89b20599a239aa5bb 2023-08-18 03:00:40,718 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/6889c9b9c4d94cb087ede86c452b05bc to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/6889c9b9c4d94cb087ede86c452b05bc 2023-08-18 03:00:40,719 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/0a837b468fef4dbba4426dc5ebb78ad6 to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/0a837b468fef4dbba4426dc5ebb78ad6 2023-08-18 03:00:40,720 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/a5f44d8b95e740a287a5f4b8fac625a6 to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/a5f44d8b95e740a287a5f4b8fac625a6 2023-08-18 03:00:40,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=1 2023-08-18 03:00:40,726 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. 2023-08-18 03:00:40,726 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a65569a4fb904c2acd072fd335815f9a: 2023-08-18 03:00:40,728 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:40,729 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=a65569a4fb904c2acd072fd335815f9a, regionState=CLOSED 2023-08-18 03:00:40,730 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1692327640729"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692327640729"}]},"ts":"1692327640729"} 2023-08-18 03:00:40,733 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-08-18 03:00:40,733 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure a65569a4fb904c2acd072fd335815f9a, server=jenkins-hbase4.apache.org,38805,1692327604836 in 209 msec 2023-08-18 03:00:40,735 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-08-18 03:00:40,735 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=a65569a4fb904c2acd072fd335815f9a, UNASSIGN in 215 msec 2023-08-18 03:00:40,746 INFO [PEWorker-5] assignment.SplitTableRegionProcedure(694): pid=12 splitting 2 storefiles, region=a65569a4fb904c2acd072fd335815f9a, threads=2 2023-08-18 03:00:40,747 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4eefa1f4973a48f89640518373c5e383 for region: a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:40,747 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/ca65f4e0a1394a7997b39f3079177db1 for region: a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:40,758 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/ca65f4e0a1394a7997b39f3079177db1, top=true 2023-08-18 03:00:40,769 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/.splits/1ff6f8076f79980eeb8f20457c47fb98/info/TestLogRolling-testLogRolling=a65569a4fb904c2acd072fd335815f9a-ca65f4e0a1394a7997b39f3079177db1 for child: 1ff6f8076f79980eeb8f20457c47fb98, parent: a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:40,769 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/ca65f4e0a1394a7997b39f3079177db1 for region: a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:40,785 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4eefa1f4973a48f89640518373c5e383 for region: a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:00:40,786 DEBUG [PEWorker-5] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region a65569a4fb904c2acd072fd335815f9a Daughter A: 1 storefiles, Daughter B: 2 storefiles. 2023-08-18 03:00:40,818 DEBUG [PEWorker-5] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a9284c12aea305493291bf3e413c6c3d/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=-1 2023-08-18 03:00:40,820 DEBUG [PEWorker-5] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=-1 2023-08-18 03:00:40,822 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1692327640822"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1692327640822"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1692327640822"}]},"ts":"1692327640822"} 2023-08-18 03:00:40,822 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692327640822"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692327640822"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692327640822"}]},"ts":"1692327640822"} 2023-08-18 03:00:40,822 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692327640822"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692327640822"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692327640822"}]},"ts":"1692327640822"} 2023-08-18 03:00:40,862 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38805] regionserver.HRegion(9158): Flush requested on 1588230740 2023-08-18 03:00:40,862 DEBUG [MemStoreFlusher.0] regionserver.FlushAllLargeStoresPolicy(69): Since none of the CFs were above the size, flushing all. 2023-08-18 03:00:40,862 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=4.82 KB heapSize=8.36 KB 2023-08-18 03:00:40,871 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=a9284c12aea305493291bf3e413c6c3d, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=1ff6f8076f79980eeb8f20457c47fb98, ASSIGN}] 2023-08-18 03:00:40,873 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=a9284c12aea305493291bf3e413c6c3d, ASSIGN 2023-08-18 03:00:40,873 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=1ff6f8076f79980eeb8f20457c47fb98, ASSIGN 2023-08-18 03:00:40,875 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=a9284c12aea305493291bf3e413c6c3d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38805,1692327604836; forceNewPlan=false, retain=false 2023-08-18 03:00:40,875 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=1ff6f8076f79980eeb8f20457c47fb98, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38805,1692327604836; forceNewPlan=false, retain=false 2023-08-18 03:00:40,876 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.61 KB at sequenceid=17 (bloomFilter=false), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/.tmp/info/10f27a576ede4cd785d7d12bbc999c25 2023-08-18 03:00:40,890 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=216 B at sequenceid=17 (bloomFilter=false), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/.tmp/table/91df80723fd44d61994e12852be67142 2023-08-18 03:00:40,896 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/.tmp/info/10f27a576ede4cd785d7d12bbc999c25 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/info/10f27a576ede4cd785d7d12bbc999c25 2023-08-18 03:00:40,900 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/info/10f27a576ede4cd785d7d12bbc999c25, entries=29, sequenceid=17, filesize=8.6 K 2023-08-18 03:00:40,901 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/.tmp/table/91df80723fd44d61994e12852be67142 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/table/91df80723fd44d61994e12852be67142 2023-08-18 03:00:40,906 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/table/91df80723fd44d61994e12852be67142, entries=4, sequenceid=17, filesize=4.8 K 2023-08-18 03:00:40,907 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~4.82 KB/4934, heapSize ~8.08 KB/8272, currentSize=0 B/0 for 1588230740 in 45ms, sequenceid=17, compaction requested=false 2023-08-18 03:00:40,907 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-08-18 03:00:41,026 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=1ff6f8076f79980eeb8f20457c47fb98, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:41,026 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=a9284c12aea305493291bf3e413c6c3d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:41,026 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692327641026"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692327641026"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692327641026"}]},"ts":"1692327641026"} 2023-08-18 03:00:41,026 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692327641026"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692327641026"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692327641026"}]},"ts":"1692327641026"} 2023-08-18 03:00:41,028 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure 1ff6f8076f79980eeb8f20457c47fb98, server=jenkins-hbase4.apache.org,38805,1692327604836}] 2023-08-18 03:00:41,029 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure a9284c12aea305493291bf3e413c6c3d, server=jenkins-hbase4.apache.org,38805,1692327604836}] 2023-08-18 03:00:41,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d. 2023-08-18 03:00:41,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a9284c12aea305493291bf3e413c6c3d, NAME => 'TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d.', STARTKEY => '', ENDKEY => 'row0064'} 2023-08-18 03:00:41,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling a9284c12aea305493291bf3e413c6c3d 2023-08-18 03:00:41,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:00:41,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a9284c12aea305493291bf3e413c6c3d 2023-08-18 03:00:41,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a9284c12aea305493291bf3e413c6c3d 2023-08-18 03:00:41,186 INFO [StoreOpener-a9284c12aea305493291bf3e413c6c3d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region a9284c12aea305493291bf3e413c6c3d 2023-08-18 03:00:41,187 DEBUG [StoreOpener-a9284c12aea305493291bf3e413c6c3d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a9284c12aea305493291bf3e413c6c3d/info 2023-08-18 03:00:41,187 DEBUG [StoreOpener-a9284c12aea305493291bf3e413c6c3d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a9284c12aea305493291bf3e413c6c3d/info 2023-08-18 03:00:41,187 INFO [StoreOpener-a9284c12aea305493291bf3e413c6c3d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9284c12aea305493291bf3e413c6c3d columnFamilyName info 2023-08-18 03:00:41,201 DEBUG [StoreOpener-a9284c12aea305493291bf3e413c6c3d-1] regionserver.HStore(539): loaded hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a9284c12aea305493291bf3e413c6c3d/info/4eefa1f4973a48f89640518373c5e383.a65569a4fb904c2acd072fd335815f9a->hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4eefa1f4973a48f89640518373c5e383-bottom 2023-08-18 03:00:41,202 INFO [StoreOpener-a9284c12aea305493291bf3e413c6c3d-1] regionserver.HStore(310): Store=a9284c12aea305493291bf3e413c6c3d/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:00:41,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a9284c12aea305493291bf3e413c6c3d 2023-08-18 03:00:41,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a9284c12aea305493291bf3e413c6c3d 2023-08-18 03:00:41,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a9284c12aea305493291bf3e413c6c3d 2023-08-18 03:00:41,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a9284c12aea305493291bf3e413c6c3d; next sequenceid=114; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=878089, jitterRate=0.11654812097549438}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-18 03:00:41,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a9284c12aea305493291bf3e413c6c3d: 2023-08-18 03:00:41,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d., pid=18, masterSystemTime=1692327641180 2023-08-18 03:00:41,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:00:41,212 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-08-18 03:00:41,213 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d. 2023-08-18 03:00:41,213 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1912): a9284c12aea305493291bf3e413c6c3d/info is initiating minor compaction (all files) 2023-08-18 03:00:41,213 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a9284c12aea305493291bf3e413c6c3d/info in TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d. 2023-08-18 03:00:41,213 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a9284c12aea305493291bf3e413c6c3d/info/4eefa1f4973a48f89640518373c5e383.a65569a4fb904c2acd072fd335815f9a->hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4eefa1f4973a48f89640518373c5e383-bottom] into tmpdir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a9284c12aea305493291bf3e413c6c3d/.tmp, totalSize=75.0 K 2023-08-18 03:00:41,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d. 2023-08-18 03:00:41,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d. 2023-08-18 03:00:41,214 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting 4eefa1f4973a48f89640518373c5e383.a65569a4fb904c2acd072fd335815f9a, keycount=34, bloomtype=ROW, size=75.0 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1692327616191 2023-08-18 03:00:41,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:00:41,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1ff6f8076f79980eeb8f20457c47fb98, NAME => 'TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98.', STARTKEY => 'row0064', ENDKEY => ''} 2023-08-18 03:00:41,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:00:41,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:00:41,216 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=a9284c12aea305493291bf3e413c6c3d, regionState=OPEN, openSeqNum=114, regionLocation=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:41,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:00:41,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:00:41,216 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692327641216"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692327641216"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692327641216"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692327641216"}]},"ts":"1692327641216"} 2023-08-18 03:00:41,221 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-08-18 03:00:41,221 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure a9284c12aea305493291bf3e413c6c3d, server=jenkins-hbase4.apache.org,38805,1692327604836 in 189 msec 2023-08-18 03:00:41,221 INFO [StoreOpener-1ff6f8076f79980eeb8f20457c47fb98-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:00:41,223 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=a9284c12aea305493291bf3e413c6c3d, ASSIGN in 350 msec 2023-08-18 03:00:41,225 DEBUG [StoreOpener-1ff6f8076f79980eeb8f20457c47fb98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info 2023-08-18 03:00:41,225 DEBUG [StoreOpener-1ff6f8076f79980eeb8f20457c47fb98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info 2023-08-18 03:00:41,226 INFO [StoreOpener-1ff6f8076f79980eeb8f20457c47fb98-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1ff6f8076f79980eeb8f20457c47fb98 columnFamilyName info 2023-08-18 03:00:41,228 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] throttle.PressureAwareThroughputController(145): a9284c12aea305493291bf3e413c6c3d#info#compaction#25 average throughput is 21.05 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-18 03:00:41,238 DEBUG [StoreOpener-1ff6f8076f79980eeb8f20457c47fb98-1] regionserver.HStore(539): loaded hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/4eefa1f4973a48f89640518373c5e383.a65569a4fb904c2acd072fd335815f9a->hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4eefa1f4973a48f89640518373c5e383-top 2023-08-18 03:00:41,250 DEBUG [StoreOpener-1ff6f8076f79980eeb8f20457c47fb98-1] regionserver.HStore(539): loaded hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/TestLogRolling-testLogRolling=a65569a4fb904c2acd072fd335815f9a-ca65f4e0a1394a7997b39f3079177db1 2023-08-18 03:00:41,251 INFO [StoreOpener-1ff6f8076f79980eeb8f20457c47fb98-1] regionserver.HStore(310): Store=1ff6f8076f79980eeb8f20457c47fb98/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:00:41,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:00:41,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:00:41,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:00:41,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1ff6f8076f79980eeb8f20457c47fb98; next sequenceid=114; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=854801, jitterRate=0.08693648874759674}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-18 03:00:41,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:00:41,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98., pid=17, masterSystemTime=1692327641180 2023-08-18 03:00:41,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-18 03:00:41,263 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-08-18 03:00:41,264 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a9284c12aea305493291bf3e413c6c3d/.tmp/info/2a5075c5fee941bd86d49761ae3ab1dd as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a9284c12aea305493291bf3e413c6c3d/info/2a5075c5fee941bd86d49761ae3ab1dd 2023-08-18 03:00:41,264 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:00:41,264 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HStore(1912): 1ff6f8076f79980eeb8f20457c47fb98/info is initiating minor compaction (all files) 2023-08-18 03:00:41,265 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 1ff6f8076f79980eeb8f20457c47fb98/info in TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:00:41,265 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/4eefa1f4973a48f89640518373c5e383.a65569a4fb904c2acd072fd335815f9a->hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4eefa1f4973a48f89640518373c5e383-top, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/TestLogRolling-testLogRolling=a65569a4fb904c2acd072fd335815f9a-ca65f4e0a1394a7997b39f3079177db1] into tmpdir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp, totalSize=101.3 K 2023-08-18 03:00:41,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:00:41,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:00:41,267 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] compactions.Compactor(207): Compacting 4eefa1f4973a48f89640518373c5e383.a65569a4fb904c2acd072fd335815f9a, keycount=34, bloomtype=ROW, size=75.0 K, encoding=NONE, compression=NONE, seqNum=86, earliestPutTs=1692327616191 2023-08-18 03:00:41,268 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=1ff6f8076f79980eeb8f20457c47fb98, regionState=OPEN, openSeqNum=114, regionLocation=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:00:41,268 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692327641268"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692327641268"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692327641268"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692327641268"}]},"ts":"1692327641268"} 2023-08-18 03:00:41,270 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] compactions.Compactor(207): Compacting TestLogRolling-testLogRolling=a65569a4fb904c2acd072fd335815f9a-ca65f4e0a1394a7997b39f3079177db1, keycount=21, bloomtype=ROW, size=26.3 K, encoding=NONE, compression=NONE, seqNum=110, earliestPutTs=1692327640427 2023-08-18 03:00:41,277 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRolling' 2023-08-18 03:00:41,278 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-08-18 03:00:41,278 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure 1ff6f8076f79980eeb8f20457c47fb98, server=jenkins-hbase4.apache.org,38805,1692327604836 in 243 msec 2023-08-18 03:00:41,279 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 1 (all) file(s) in a9284c12aea305493291bf3e413c6c3d/info of a9284c12aea305493291bf3e413c6c3d into 2a5075c5fee941bd86d49761ae3ab1dd(size=69.7 K), total size for store is 69.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-18 03:00:41,279 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a9284c12aea305493291bf3e413c6c3d: 2023-08-18 03:00:41,279 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d., storeName=a9284c12aea305493291bf3e413c6c3d/info, priority=15, startTime=1692327641211; duration=0sec 2023-08-18 03:00:41,279 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:00:41,281 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-08-18 03:00:41,281 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=1ff6f8076f79980eeb8f20457c47fb98, ASSIGN in 407 msec 2023-08-18 03:00:41,283 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=a65569a4fb904c2acd072fd335815f9a, daughterA=a9284c12aea305493291bf3e413c6c3d, daughterB=1ff6f8076f79980eeb8f20457c47fb98 in 777 msec 2023-08-18 03:00:41,284 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] throttle.PressureAwareThroughputController(145): 1ff6f8076f79980eeb8f20457c47fb98#info#compaction#26 average throughput is 26.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-18 03:00:41,303 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/fbbd5501476b43f2a3f21e1302cf967d as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/fbbd5501476b43f2a3f21e1302cf967d 2023-08-18 03:00:41,311 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HStore(1652): Completed compaction of 2 (all) file(s) in 1ff6f8076f79980eeb8f20457c47fb98/info of 1ff6f8076f79980eeb8f20457c47fb98 into fbbd5501476b43f2a3f21e1302cf967d(size=31.6 K), total size for store is 31.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-18 03:00:41,311 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:00:41,311 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98., storeName=1ff6f8076f79980eeb8f20457c47fb98/info, priority=14, startTime=1692327641262; duration=0sec 2023-08-18 03:00:41,311 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:00:46,304 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-18 03:00:50,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] ipc.CallRunner(144): callId: 104 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:38982 deadline: 1692327660482, exception=org.apache.hadoop.hbase.NotServingRegionException: TestLogRolling-testLogRolling,,1692327606176.a65569a4fb904c2acd072fd335815f9a. is not online on jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:01:12,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:01:12,500 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1ff6f8076f79980eeb8f20457c47fb98 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-18 03:01:12,516 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=125 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/937e9e5742f1438fa1c4ddbad6e3bce1 2023-08-18 03:01:12,522 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/937e9e5742f1438fa1c4ddbad6e3bce1 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/937e9e5742f1438fa1c4ddbad6e3bce1 2023-08-18 03:01:12,530 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=1ff6f8076f79980eeb8f20457c47fb98, server=jenkins-hbase4.apache.org,38805,1692327604836 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-18 03:01:12,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] ipc.CallRunner(144): callId: 137 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:38982 deadline: 1692327682530, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=1ff6f8076f79980eeb8f20457c47fb98, server=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:01:12,531 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/937e9e5742f1438fa1c4ddbad6e3bce1, entries=8, sequenceid=125, filesize=13.0 K 2023-08-18 03:01:12,533 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 1ff6f8076f79980eeb8f20457c47fb98 in 33ms, sequenceid=125, compaction requested=false 2023-08-18 03:01:12,533 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:01:22,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:01:22,602 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1ff6f8076f79980eeb8f20457c47fb98 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-18 03:01:22,615 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=1ff6f8076f79980eeb8f20457c47fb98, server=jenkins-hbase4.apache.org,38805,1692327604836 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-18 03:01:22,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] ipc.CallRunner(144): callId: 147 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:38982 deadline: 1692327692615, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=1ff6f8076f79980eeb8f20457c47fb98, server=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:01:22,616 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=150 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/497addb6a6e34db3bd106e6d446f751a 2023-08-18 03:01:22,624 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/497addb6a6e34db3bd106e6d446f751a as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/497addb6a6e34db3bd106e6d446f751a 2023-08-18 03:01:22,629 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/497addb6a6e34db3bd106e6d446f751a, entries=22, sequenceid=150, filesize=27.4 K 2023-08-18 03:01:22,629 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=7.18 KB/7357 for 1ff6f8076f79980eeb8f20457c47fb98 in 27ms, sequenceid=150, compaction requested=true 2023-08-18 03:01:22,630 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:01:22,630 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:01:22,630 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-18 03:01:22,631 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 73719 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-18 03:01:22,631 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1912): 1ff6f8076f79980eeb8f20457c47fb98/info is initiating minor compaction (all files) 2023-08-18 03:01:22,631 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1ff6f8076f79980eeb8f20457c47fb98/info in TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:01:22,631 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/fbbd5501476b43f2a3f21e1302cf967d, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/937e9e5742f1438fa1c4ddbad6e3bce1, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/497addb6a6e34db3bd106e6d446f751a] into tmpdir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp, totalSize=72.0 K 2023-08-18 03:01:22,631 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting fbbd5501476b43f2a3f21e1302cf967d, keycount=26, bloomtype=ROW, size=31.6 K, encoding=NONE, compression=NONE, seqNum=110, earliestPutTs=1692327638420 2023-08-18 03:01:22,632 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting 937e9e5742f1438fa1c4ddbad6e3bce1, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=125, earliestPutTs=1692327670491 2023-08-18 03:01:22,632 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting 497addb6a6e34db3bd106e6d446f751a, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=150, earliestPutTs=1692327672501 2023-08-18 03:01:22,643 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1ff6f8076f79980eeb8f20457c47fb98#info#compaction#29 average throughput is 56.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-18 03:01:22,660 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/28b20b4633d9457c80bbc3215e536a6b as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/28b20b4633d9457c80bbc3215e536a6b 2023-08-18 03:01:22,666 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1ff6f8076f79980eeb8f20457c47fb98/info of 1ff6f8076f79980eeb8f20457c47fb98 into 28b20b4633d9457c80bbc3215e536a6b(size=62.7 K), total size for store is 62.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-18 03:01:22,666 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:01:22,666 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98., storeName=1ff6f8076f79980eeb8f20457c47fb98/info, priority=13, startTime=1692327682630; duration=0sec 2023-08-18 03:01:22,666 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:01:32,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:01:32,667 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1ff6f8076f79980eeb8f20457c47fb98 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-18 03:01:32,680 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=162 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/d4d41dd35e37494f9993ed8ab36e08cb 2023-08-18 03:01:32,687 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/d4d41dd35e37494f9993ed8ab36e08cb as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/d4d41dd35e37494f9993ed8ab36e08cb 2023-08-18 03:01:32,692 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/d4d41dd35e37494f9993ed8ab36e08cb, entries=8, sequenceid=162, filesize=13.0 K 2023-08-18 03:01:32,693 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=1.03 KB/1051 for 1ff6f8076f79980eeb8f20457c47fb98 in 26ms, sequenceid=162, compaction requested=false 2023-08-18 03:01:32,693 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:01:34,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:01:34,678 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1ff6f8076f79980eeb8f20457c47fb98 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-18 03:01:34,695 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=173 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/ce53a364e651446fbe3bf182f9271d5a 2023-08-18 03:01:34,701 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/ce53a364e651446fbe3bf182f9271d5a as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/ce53a364e651446fbe3bf182f9271d5a 2023-08-18 03:01:34,705 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=1ff6f8076f79980eeb8f20457c47fb98, server=jenkins-hbase4.apache.org,38805,1692327604836 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-18 03:01:34,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] ipc.CallRunner(144): callId: 179 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:38982 deadline: 1692327704705, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=1ff6f8076f79980eeb8f20457c47fb98, server=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:01:34,706 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/ce53a364e651446fbe3bf182f9271d5a, entries=8, sequenceid=173, filesize=13.0 K 2023-08-18 03:01:34,707 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 1ff6f8076f79980eeb8f20457c47fb98 in 29ms, sequenceid=173, compaction requested=true 2023-08-18 03:01:34,707 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:01:34,707 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:01:34,707 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-18 03:01:34,708 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 90753 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-18 03:01:34,708 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1912): 1ff6f8076f79980eeb8f20457c47fb98/info is initiating minor compaction (all files) 2023-08-18 03:01:34,709 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1ff6f8076f79980eeb8f20457c47fb98/info in TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:01:34,709 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/28b20b4633d9457c80bbc3215e536a6b, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/d4d41dd35e37494f9993ed8ab36e08cb, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/ce53a364e651446fbe3bf182f9271d5a] into tmpdir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp, totalSize=88.6 K 2023-08-18 03:01:34,709 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting 28b20b4633d9457c80bbc3215e536a6b, keycount=56, bloomtype=ROW, size=62.7 K, encoding=NONE, compression=NONE, seqNum=150, earliestPutTs=1692327638420 2023-08-18 03:01:34,709 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting d4d41dd35e37494f9993ed8ab36e08cb, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=162, earliestPutTs=1692327682603 2023-08-18 03:01:34,710 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting ce53a364e651446fbe3bf182f9271d5a, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=173, earliestPutTs=1692327692668 2023-08-18 03:01:34,723 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1ff6f8076f79980eeb8f20457c47fb98#info#compaction#32 average throughput is 18.04 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-18 03:01:34,738 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/18635ba09afe49e08eab67e1735235a9 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/18635ba09afe49e08eab67e1735235a9 2023-08-18 03:01:34,745 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1ff6f8076f79980eeb8f20457c47fb98/info of 1ff6f8076f79980eeb8f20457c47fb98 into 18635ba09afe49e08eab67e1735235a9(size=79.3 K), total size for store is 79.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-18 03:01:34,745 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:01:34,745 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98., storeName=1ff6f8076f79980eeb8f20457c47fb98/info, priority=13, startTime=1692327694707; duration=0sec 2023-08-18 03:01:34,746 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:01:44,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:01:44,720 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1ff6f8076f79980eeb8f20457c47fb98 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-18 03:01:44,740 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=199 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/1ad75ee0d2c24786957b8117df3e3341 2023-08-18 03:01:44,746 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/1ad75ee0d2c24786957b8117df3e3341 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/1ad75ee0d2c24786957b8117df3e3341 2023-08-18 03:01:44,752 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/1ad75ee0d2c24786957b8117df3e3341, entries=22, sequenceid=199, filesize=27.4 K 2023-08-18 03:01:44,753 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=3.08 KB/3153 for 1ff6f8076f79980eeb8f20457c47fb98 in 33ms, sequenceid=199, compaction requested=false 2023-08-18 03:01:44,753 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:01:46,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:01:46,737 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1ff6f8076f79980eeb8f20457c47fb98 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-18 03:01:46,778 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=1ff6f8076f79980eeb8f20457c47fb98, server=jenkins-hbase4.apache.org,38805,1692327604836 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-18 03:01:46,778 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=210 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/71dd9a5b52e44be092be0f28645b2aab 2023-08-18 03:01:46,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] ipc.CallRunner(144): callId: 211 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:38982 deadline: 1692327716778, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=1ff6f8076f79980eeb8f20457c47fb98, server=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:01:46,786 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/71dd9a5b52e44be092be0f28645b2aab as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/71dd9a5b52e44be092be0f28645b2aab 2023-08-18 03:01:46,792 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/71dd9a5b52e44be092be0f28645b2aab, entries=8, sequenceid=210, filesize=13.0 K 2023-08-18 03:01:46,793 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 1ff6f8076f79980eeb8f20457c47fb98 in 56ms, sequenceid=210, compaction requested=true 2023-08-18 03:01:46,793 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:01:46,793 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:01:46,793 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-18 03:01:46,795 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 122538 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-18 03:01:46,795 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1912): 1ff6f8076f79980eeb8f20457c47fb98/info is initiating minor compaction (all files) 2023-08-18 03:01:46,795 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1ff6f8076f79980eeb8f20457c47fb98/info in TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:01:46,795 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/18635ba09afe49e08eab67e1735235a9, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/1ad75ee0d2c24786957b8117df3e3341, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/71dd9a5b52e44be092be0f28645b2aab] into tmpdir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp, totalSize=119.7 K 2023-08-18 03:01:46,796 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting 18635ba09afe49e08eab67e1735235a9, keycount=72, bloomtype=ROW, size=79.3 K, encoding=NONE, compression=NONE, seqNum=173, earliestPutTs=1692327638420 2023-08-18 03:01:46,796 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting 1ad75ee0d2c24786957b8117df3e3341, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=199, earliestPutTs=1692327694679 2023-08-18 03:01:46,797 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting 71dd9a5b52e44be092be0f28645b2aab, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=210, earliestPutTs=1692327704721 2023-08-18 03:01:46,824 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1ff6f8076f79980eeb8f20457c47fb98#info#compaction#35 average throughput is 34.08 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-18 03:01:46,858 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/293a44c7a4014ac7828712f83c8ff712 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/293a44c7a4014ac7828712f83c8ff712 2023-08-18 03:01:46,869 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1ff6f8076f79980eeb8f20457c47fb98/info of 1ff6f8076f79980eeb8f20457c47fb98 into 293a44c7a4014ac7828712f83c8ff712(size=110.3 K), total size for store is 110.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-18 03:01:46,869 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:01:46,869 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98., storeName=1ff6f8076f79980eeb8f20457c47fb98/info, priority=13, startTime=1692327706793; duration=0sec 2023-08-18 03:01:46,869 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:01:56,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:01:56,825 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1ff6f8076f79980eeb8f20457c47fb98 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-18 03:01:56,841 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=236 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/a00069d50e924683a5b31de82bed4a6f 2023-08-18 03:01:56,853 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/a00069d50e924683a5b31de82bed4a6f as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/a00069d50e924683a5b31de82bed4a6f 2023-08-18 03:01:56,859 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/a00069d50e924683a5b31de82bed4a6f, entries=22, sequenceid=236, filesize=27.4 K 2023-08-18 03:01:56,861 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=5.13 KB/5255 for 1ff6f8076f79980eeb8f20457c47fb98 in 35ms, sequenceid=236, compaction requested=false 2023-08-18 03:01:56,861 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:01:58,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:01:58,844 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1ff6f8076f79980eeb8f20457c47fb98 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-18 03:01:58,864 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=247 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/d080d503a8c64cc399b9e1af31efec2a 2023-08-18 03:01:58,873 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/d080d503a8c64cc399b9e1af31efec2a as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/d080d503a8c64cc399b9e1af31efec2a 2023-08-18 03:01:58,881 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/d080d503a8c64cc399b9e1af31efec2a, entries=8, sequenceid=247, filesize=13.0 K 2023-08-18 03:01:58,882 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=22.58 KB/23122 for 1ff6f8076f79980eeb8f20457c47fb98 in 39ms, sequenceid=247, compaction requested=true 2023-08-18 03:01:58,883 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:01:58,883 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-18 03:01:58,883 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-18 03:01:58,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:01:58,884 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1ff6f8076f79980eeb8f20457c47fb98 1/1 column families, dataSize=23.61 KB heapSize=25.59 KB 2023-08-18 03:01:58,888 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 154238 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-18 03:01:58,888 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HStore(1912): 1ff6f8076f79980eeb8f20457c47fb98/info is initiating minor compaction (all files) 2023-08-18 03:01:58,888 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 1ff6f8076f79980eeb8f20457c47fb98/info in TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:01:58,888 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/293a44c7a4014ac7828712f83c8ff712, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/a00069d50e924683a5b31de82bed4a6f, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/d080d503a8c64cc399b9e1af31efec2a] into tmpdir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp, totalSize=150.6 K 2023-08-18 03:01:58,889 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] compactions.Compactor(207): Compacting 293a44c7a4014ac7828712f83c8ff712, keycount=102, bloomtype=ROW, size=110.3 K, encoding=NONE, compression=NONE, seqNum=210, earliestPutTs=1692327638420 2023-08-18 03:01:58,890 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] compactions.Compactor(207): Compacting a00069d50e924683a5b31de82bed4a6f, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=236, earliestPutTs=1692327706738 2023-08-18 03:01:58,890 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] compactions.Compactor(207): Compacting d080d503a8c64cc399b9e1af31efec2a, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=247, earliestPutTs=1692327716826 2023-08-18 03:01:58,916 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=23.61 KB at sequenceid=273 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/9f3845bbdb914aaa835e66536b3a8fc3 2023-08-18 03:01:58,924 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] throttle.PressureAwareThroughputController(145): 1ff6f8076f79980eeb8f20457c47fb98#info#compaction#39 average throughput is 66.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-18 03:01:58,933 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/9f3845bbdb914aaa835e66536b3a8fc3 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/9f3845bbdb914aaa835e66536b3a8fc3 2023-08-18 03:01:58,947 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/9f3845bbdb914aaa835e66536b3a8fc3, entries=23, sequenceid=273, filesize=28.4 K 2023-08-18 03:01:58,948 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~23.61 KB/24173, heapSize ~25.57 KB/26184, currentSize=6.16 KB/6306 for 1ff6f8076f79980eeb8f20457c47fb98 in 65ms, sequenceid=273, compaction requested=false 2023-08-18 03:01:58,948 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:01:58,955 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/9141f053834c47638c2665510463c50e as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/9141f053834c47638c2665510463c50e 2023-08-18 03:01:58,961 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1ff6f8076f79980eeb8f20457c47fb98/info of 1ff6f8076f79980eeb8f20457c47fb98 into 9141f053834c47638c2665510463c50e(size=141.4 K), total size for store is 169.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-18 03:01:58,961 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:01:58,961 INFO [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98., storeName=1ff6f8076f79980eeb8f20457c47fb98/info, priority=13, startTime=1692327718883; duration=0sec 2023-08-18 03:01:58,962 DEBUG [RS:0;jenkins-hbase4:38805-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:02:00,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:02:00,897 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1ff6f8076f79980eeb8f20457c47fb98 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-18 03:02:00,918 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=285 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/0ef6f0e5f61a4e71a7f148eefc13e5fe 2023-08-18 03:02:00,934 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/0ef6f0e5f61a4e71a7f148eefc13e5fe as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/0ef6f0e5f61a4e71a7f148eefc13e5fe 2023-08-18 03:02:00,940 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=1ff6f8076f79980eeb8f20457c47fb98, server=jenkins-hbase4.apache.org,38805,1692327604836 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-18 03:02:00,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] ipc.CallRunner(144): callId: 274 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:38982 deadline: 1692327730940, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=1ff6f8076f79980eeb8f20457c47fb98, server=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:02:00,947 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/0ef6f0e5f61a4e71a7f148eefc13e5fe, entries=8, sequenceid=285, filesize=13.0 K 2023-08-18 03:02:00,948 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 1ff6f8076f79980eeb8f20457c47fb98 in 51ms, sequenceid=285, compaction requested=true 2023-08-18 03:02:00,948 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:02:00,948 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:02:00,948 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-18 03:02:00,950 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 187211 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-18 03:02:00,950 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1912): 1ff6f8076f79980eeb8f20457c47fb98/info is initiating minor compaction (all files) 2023-08-18 03:02:00,950 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1ff6f8076f79980eeb8f20457c47fb98/info in TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:02:00,950 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/9141f053834c47638c2665510463c50e, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/9f3845bbdb914aaa835e66536b3a8fc3, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/0ef6f0e5f61a4e71a7f148eefc13e5fe] into tmpdir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp, totalSize=182.8 K 2023-08-18 03:02:00,950 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting 9141f053834c47638c2665510463c50e, keycount=132, bloomtype=ROW, size=141.4 K, encoding=NONE, compression=NONE, seqNum=247, earliestPutTs=1692327638420 2023-08-18 03:02:00,951 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting 9f3845bbdb914aaa835e66536b3a8fc3, keycount=23, bloomtype=ROW, size=28.4 K, encoding=NONE, compression=NONE, seqNum=273, earliestPutTs=1692327718844 2023-08-18 03:02:00,951 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting 0ef6f0e5f61a4e71a7f148eefc13e5fe, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=285, earliestPutTs=1692327718885 2023-08-18 03:02:00,966 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1ff6f8076f79980eeb8f20457c47fb98#info#compaction#41 average throughput is 40.84 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-18 03:02:00,985 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/bce4132d5d5c4a2d83b66f4cd79f120a as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/bce4132d5d5c4a2d83b66f4cd79f120a 2023-08-18 03:02:00,993 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1ff6f8076f79980eeb8f20457c47fb98/info of 1ff6f8076f79980eeb8f20457c47fb98 into bce4132d5d5c4a2d83b66f4cd79f120a(size=173.4 K), total size for store is 173.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-18 03:02:00,993 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:02:00,993 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98., storeName=1ff6f8076f79980eeb8f20457c47fb98/info, priority=13, startTime=1692327720948; duration=0sec 2023-08-18 03:02:00,993 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:02:05,210 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/4eefa1f4973a48f89640518373c5e383.a65569a4fb904c2acd072fd335815f9a->hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4eefa1f4973a48f89640518373c5e383-top, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/fbbd5501476b43f2a3f21e1302cf967d, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/TestLogRolling-testLogRolling=a65569a4fb904c2acd072fd335815f9a-ca65f4e0a1394a7997b39f3079177db1, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/937e9e5742f1438fa1c4ddbad6e3bce1, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/28b20b4633d9457c80bbc3215e536a6b, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/497addb6a6e34db3bd106e6d446f751a, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/d4d41dd35e37494f9993ed8ab36e08cb, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/18635ba09afe49e08eab67e1735235a9, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/ce53a364e651446fbe3bf182f9271d5a, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/1ad75ee0d2c24786957b8117df3e3341, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/293a44c7a4014ac7828712f83c8ff712, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/71dd9a5b52e44be092be0f28645b2aab, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/a00069d50e924683a5b31de82bed4a6f, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/9141f053834c47638c2665510463c50e, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/d080d503a8c64cc399b9e1af31efec2a, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/9f3845bbdb914aaa835e66536b3a8fc3, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/0ef6f0e5f61a4e71a7f148eefc13e5fe] to archive 2023-08-18 03:02:05,213 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-08-18 03:02:05,215 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/4eefa1f4973a48f89640518373c5e383.a65569a4fb904c2acd072fd335815f9a to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/4eefa1f4973a48f89640518373c5e383.a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:02:05,216 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/fbbd5501476b43f2a3f21e1302cf967d to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/fbbd5501476b43f2a3f21e1302cf967d 2023-08-18 03:02:05,218 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/TestLogRolling-testLogRolling=a65569a4fb904c2acd072fd335815f9a-ca65f4e0a1394a7997b39f3079177db1 to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/TestLogRolling-testLogRolling=a65569a4fb904c2acd072fd335815f9a-ca65f4e0a1394a7997b39f3079177db1 2023-08-18 03:02:05,219 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/937e9e5742f1438fa1c4ddbad6e3bce1 to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/937e9e5742f1438fa1c4ddbad6e3bce1 2023-08-18 03:02:05,220 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/28b20b4633d9457c80bbc3215e536a6b to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/28b20b4633d9457c80bbc3215e536a6b 2023-08-18 03:02:05,222 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/497addb6a6e34db3bd106e6d446f751a to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/497addb6a6e34db3bd106e6d446f751a 2023-08-18 03:02:05,223 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/d4d41dd35e37494f9993ed8ab36e08cb to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/d4d41dd35e37494f9993ed8ab36e08cb 2023-08-18 03:02:05,224 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/18635ba09afe49e08eab67e1735235a9 to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/18635ba09afe49e08eab67e1735235a9 2023-08-18 03:02:05,226 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/ce53a364e651446fbe3bf182f9271d5a to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/ce53a364e651446fbe3bf182f9271d5a 2023-08-18 03:02:05,227 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/1ad75ee0d2c24786957b8117df3e3341 to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/1ad75ee0d2c24786957b8117df3e3341 2023-08-18 03:02:05,228 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/293a44c7a4014ac7828712f83c8ff712 to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/293a44c7a4014ac7828712f83c8ff712 2023-08-18 03:02:05,229 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/71dd9a5b52e44be092be0f28645b2aab to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/71dd9a5b52e44be092be0f28645b2aab 2023-08-18 03:02:05,230 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/a00069d50e924683a5b31de82bed4a6f to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/a00069d50e924683a5b31de82bed4a6f 2023-08-18 03:02:05,231 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/9141f053834c47638c2665510463c50e to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/9141f053834c47638c2665510463c50e 2023-08-18 03:02:05,233 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/d080d503a8c64cc399b9e1af31efec2a to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/d080d503a8c64cc399b9e1af31efec2a 2023-08-18 03:02:05,234 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/9f3845bbdb914aaa835e66536b3a8fc3 to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/9f3845bbdb914aaa835e66536b3a8fc3 2023-08-18 03:02:05,235 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/0ef6f0e5f61a4e71a7f148eefc13e5fe to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/0ef6f0e5f61a4e71a7f148eefc13e5fe 2023-08-18 03:02:05,236 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a9284c12aea305493291bf3e413c6c3d/info/4eefa1f4973a48f89640518373c5e383.a65569a4fb904c2acd072fd335815f9a->hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a65569a4fb904c2acd072fd335815f9a/info/4eefa1f4973a48f89640518373c5e383-bottom] to archive 2023-08-18 03:02:05,237 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-08-18 03:02:05,238 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a9284c12aea305493291bf3e413c6c3d/info/4eefa1f4973a48f89640518373c5e383.a65569a4fb904c2acd072fd335815f9a to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/a9284c12aea305493291bf3e413c6c3d/info/4eefa1f4973a48f89640518373c5e383.a65569a4fb904c2acd072fd335815f9a 2023-08-18 03:02:11,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:02:11,017 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1ff6f8076f79980eeb8f20457c47fb98 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-18 03:02:11,028 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=1ff6f8076f79980eeb8f20457c47fb98, server=jenkins-hbase4.apache.org,38805,1692327604836 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-18 03:02:11,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] ipc.CallRunner(144): callId: 284 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:38982 deadline: 1692327741028, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=1ff6f8076f79980eeb8f20457c47fb98, server=jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:02:11,029 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=311 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/bbac1b77edf94f4680b1f095ddde99a9 2023-08-18 03:02:11,035 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/bbac1b77edf94f4680b1f095ddde99a9 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/bbac1b77edf94f4680b1f095ddde99a9 2023-08-18 03:02:11,040 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/bbac1b77edf94f4680b1f095ddde99a9, entries=22, sequenceid=311, filesize=27.4 K 2023-08-18 03:02:11,041 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=7.18 KB/7357 for 1ff6f8076f79980eeb8f20457c47fb98 in 24ms, sequenceid=311, compaction requested=false 2023-08-18 03:02:11,041 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:02:21,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38805] regionserver.HRegion(9158): Flush requested on 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:02:21,070 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1ff6f8076f79980eeb8f20457c47fb98 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-18 03:02:21,082 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=322 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/103360e757dc4024b3d5889f3bcc3a0f 2023-08-18 03:02:21,088 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/103360e757dc4024b3d5889f3bcc3a0f as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/103360e757dc4024b3d5889f3bcc3a0f 2023-08-18 03:02:21,094 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/103360e757dc4024b3d5889f3bcc3a0f, entries=8, sequenceid=322, filesize=13.0 K 2023-08-18 03:02:21,094 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=0 B/0 for 1ff6f8076f79980eeb8f20457c47fb98 in 24ms, sequenceid=322, compaction requested=true 2023-08-18 03:02:21,095 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:02:21,095 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:02:21,095 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-18 03:02:21,096 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 218941 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-18 03:02:21,096 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1912): 1ff6f8076f79980eeb8f20457c47fb98/info is initiating minor compaction (all files) 2023-08-18 03:02:21,096 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1ff6f8076f79980eeb8f20457c47fb98/info in TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:02:21,096 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/bce4132d5d5c4a2d83b66f4cd79f120a, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/bbac1b77edf94f4680b1f095ddde99a9, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/103360e757dc4024b3d5889f3bcc3a0f] into tmpdir=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp, totalSize=213.8 K 2023-08-18 03:02:21,096 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting bce4132d5d5c4a2d83b66f4cd79f120a, keycount=163, bloomtype=ROW, size=173.4 K, encoding=NONE, compression=NONE, seqNum=285, earliestPutTs=1692327638420 2023-08-18 03:02:21,097 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting bbac1b77edf94f4680b1f095ddde99a9, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=311, earliestPutTs=1692327720898 2023-08-18 03:02:21,097 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] compactions.Compactor(207): Compacting 103360e757dc4024b3d5889f3bcc3a0f, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=322, earliestPutTs=1692327731018 2023-08-18 03:02:21,110 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1ff6f8076f79980eeb8f20457c47fb98#info#compaction#44 average throughput is 64.48 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-18 03:02:21,130 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/.tmp/info/38684df167f44530b0010abada01f5c6 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/38684df167f44530b0010abada01f5c6 2023-08-18 03:02:21,136 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1ff6f8076f79980eeb8f20457c47fb98/info of 1ff6f8076f79980eeb8f20457c47fb98 into 38684df167f44530b0010abada01f5c6(size=203.8 K), total size for store is 203.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-18 03:02:21,136 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:02:21,136 INFO [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98., storeName=1ff6f8076f79980eeb8f20457c47fb98/info, priority=13, startTime=1692327741095; duration=0sec 2023-08-18 03:02:21,136 DEBUG [RS:0;jenkins-hbase4:38805-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-18 03:02:23,070 INFO [Listener at localhost/43431] wal.AbstractTestLogRolling(188): after writing there are 0 log files 2023-08-18 03:02:23,087 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33159,DS-8a5f4c93-fc1f-4509-bf62-9f3c09adff4e,DISK] 2023-08-18 03:02:23,088 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34233,DS-20db9b42-4f85-449b-8969-e172716f38b9,DISK] 2023-08-18 03:02:23,091 INFO [Listener at localhost/43431] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/WALs/jenkins-hbase4.apache.org,38805,1692327604836/jenkins-hbase4.apache.org%2C38805%2C1692327604836.1692327605359 with entries=309, filesize=300.61 KB; new WAL /user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/WALs/jenkins-hbase4.apache.org,38805,1692327604836/jenkins-hbase4.apache.org%2C38805%2C1692327604836.1692327743071 2023-08-18 03:02:23,091 DEBUG [Listener at localhost/43431] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33159,DS-8a5f4c93-fc1f-4509-bf62-9f3c09adff4e,DISK], DatanodeInfoWithStorage[127.0.0.1:34233,DS-20db9b42-4f85-449b-8969-e172716f38b9,DISK]] 2023-08-18 03:02:23,091 DEBUG [Listener at localhost/43431] wal.AbstractFSWAL(716): hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/WALs/jenkins-hbase4.apache.org,38805,1692327604836/jenkins-hbase4.apache.org%2C38805%2C1692327604836.1692327605359 is not closed yet, will try archiving it next time 2023-08-18 03:02:23,096 INFO [Listener at localhost/43431] regionserver.HRegion(2745): Flushing 6b5011f83909e605795a74cdf612ac58 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-18 03:02:23,106 INFO [Listener at localhost/43431] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/namespace/6b5011f83909e605795a74cdf612ac58/.tmp/info/f6c276d58280402d9c29d9c607c6cc09 2023-08-18 03:02:23,111 DEBUG [Listener at localhost/43431] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/namespace/6b5011f83909e605795a74cdf612ac58/.tmp/info/f6c276d58280402d9c29d9c607c6cc09 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/namespace/6b5011f83909e605795a74cdf612ac58/info/f6c276d58280402d9c29d9c607c6cc09 2023-08-18 03:02:23,115 INFO [Listener at localhost/43431] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/namespace/6b5011f83909e605795a74cdf612ac58/info/f6c276d58280402d9c29d9c607c6cc09, entries=2, sequenceid=6, filesize=4.8 K 2023-08-18 03:02:23,116 INFO [Listener at localhost/43431] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 6b5011f83909e605795a74cdf612ac58 in 20ms, sequenceid=6, compaction requested=false 2023-08-18 03:02:23,117 DEBUG [Listener at localhost/43431] regionserver.HRegion(2446): Flush status journal for 6b5011f83909e605795a74cdf612ac58: 2023-08-18 03:02:23,117 DEBUG [Listener at localhost/43431] regionserver.HRegion(2446): Flush status journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:02:23,117 INFO [Listener at localhost/43431] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.26 KB heapSize=4.19 KB 2023-08-18 03:02:23,133 INFO [Listener at localhost/43431] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.26 KB at sequenceid=24 (bloomFilter=false), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/.tmp/info/98ef8a26cb294d2f93f94a4b629e177a 2023-08-18 03:02:23,138 DEBUG [Listener at localhost/43431] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/.tmp/info/98ef8a26cb294d2f93f94a4b629e177a as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/info/98ef8a26cb294d2f93f94a4b629e177a 2023-08-18 03:02:23,143 INFO [Listener at localhost/43431] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/info/98ef8a26cb294d2f93f94a4b629e177a, entries=16, sequenceid=24, filesize=7.0 K 2023-08-18 03:02:23,143 INFO [Listener at localhost/43431] regionserver.HRegion(2948): Finished flush of dataSize ~2.26 KB/2312, heapSize ~3.67 KB/3760, currentSize=0 B/0 for 1588230740 in 26ms, sequenceid=24, compaction requested=false 2023-08-18 03:02:23,143 DEBUG [Listener at localhost/43431] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-08-18 03:02:23,144 DEBUG [Listener at localhost/43431] regionserver.HRegion(2446): Flush status journal for a9284c12aea305493291bf3e413c6c3d: 2023-08-18 03:02:23,158 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34233,DS-20db9b42-4f85-449b-8969-e172716f38b9,DISK] 2023-08-18 03:02:23,158 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33159,DS-8a5f4c93-fc1f-4509-bf62-9f3c09adff4e,DISK] 2023-08-18 03:02:23,160 INFO [Listener at localhost/43431] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/WALs/jenkins-hbase4.apache.org,38805,1692327604836/jenkins-hbase4.apache.org%2C38805%2C1692327604836.1692327743071 with entries=2, filesize=607 B; new WAL /user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/WALs/jenkins-hbase4.apache.org,38805,1692327604836/jenkins-hbase4.apache.org%2C38805%2C1692327604836.1692327743144 2023-08-18 03:02:23,160 DEBUG [Listener at localhost/43431] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34233,DS-20db9b42-4f85-449b-8969-e172716f38b9,DISK], DatanodeInfoWithStorage[127.0.0.1:33159,DS-8a5f4c93-fc1f-4509-bf62-9f3c09adff4e,DISK]] 2023-08-18 03:02:23,160 DEBUG [Listener at localhost/43431] wal.AbstractFSWAL(716): hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/WALs/jenkins-hbase4.apache.org,38805,1692327604836/jenkins-hbase4.apache.org%2C38805%2C1692327604836.1692327743071 is not closed yet, will try archiving it next time 2023-08-18 03:02:23,160 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/WALs/jenkins-hbase4.apache.org,38805,1692327604836/jenkins-hbase4.apache.org%2C38805%2C1692327604836.1692327605359 to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/oldWALs/jenkins-hbase4.apache.org%2C38805%2C1692327604836.1692327605359 2023-08-18 03:02:23,161 INFO [Listener at localhost/43431] hbase.Waiter(180): Waiting up to [5,000] milli-secs(wait.for.ratio=[1]) 2023-08-18 03:02:23,162 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/WALs/jenkins-hbase4.apache.org,38805,1692327604836/jenkins-hbase4.apache.org%2C38805%2C1692327604836.1692327743071 not finished, retry = 0 2023-08-18 03:02:23,263 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/WALs/jenkins-hbase4.apache.org,38805,1692327604836/jenkins-hbase4.apache.org%2C38805%2C1692327604836.1692327743071 to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/oldWALs/jenkins-hbase4.apache.org%2C38805%2C1692327604836.1692327743071 2023-08-18 03:02:23,361 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-18 03:02:23,361 INFO [Listener at localhost/43431] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-18 03:02:23,361 DEBUG [Listener at localhost/43431] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2630ef71 to 127.0.0.1:55029 2023-08-18 03:02:23,361 DEBUG [Listener at localhost/43431] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 03:02:23,362 DEBUG [Listener at localhost/43431] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-18 03:02:23,362 DEBUG [Listener at localhost/43431] util.JVMClusterUtil(257): Found active master hash=2067346763, stopped=false 2023-08-18 03:02:23,362 INFO [Listener at localhost/43431] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,39179,1692327604776 2023-08-18 03:02:23,363 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-18 03:02:23,363 INFO [Listener at localhost/43431] procedure2.ProcedureExecutor(629): Stopping 2023-08-18 03:02:23,363 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:02:23,363 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): regionserver:38805-0x1001d5529a90001, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-18 03:02:23,364 DEBUG [Listener at localhost/43431] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x673299c4 to 127.0.0.1:55029 2023-08-18 03:02:23,364 DEBUG [Listener at localhost/43431] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 03:02:23,364 INFO [Listener at localhost/43431] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38805,1692327604836' ***** 2023-08-18 03:02:23,364 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 03:02:23,364 INFO [Listener at localhost/43431] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-18 03:02:23,365 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38805-0x1001d5529a90001, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 03:02:23,365 INFO [RS:0;jenkins-hbase4:38805] regionserver.HeapMemoryManager(220): Stopping 2023-08-18 03:02:23,365 INFO [RS:0;jenkins-hbase4:38805] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-18 03:02:23,365 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-18 03:02:23,365 INFO [RS:0;jenkins-hbase4:38805] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-18 03:02:23,365 INFO [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(3305): Received CLOSE for 6b5011f83909e605795a74cdf612ac58 2023-08-18 03:02:23,365 INFO [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(3305): Received CLOSE for 1ff6f8076f79980eeb8f20457c47fb98 2023-08-18 03:02:23,365 INFO [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(3305): Received CLOSE for a9284c12aea305493291bf3e413c6c3d 2023-08-18 03:02:23,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6b5011f83909e605795a74cdf612ac58, disabling compactions & flushes 2023-08-18 03:02:23,365 INFO [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:02:23,365 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58. 2023-08-18 03:02:23,366 DEBUG [RS:0;jenkins-hbase4:38805] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x194ca98e to 127.0.0.1:55029 2023-08-18 03:02:23,366 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58. 2023-08-18 03:02:23,366 DEBUG [RS:0;jenkins-hbase4:38805] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 03:02:23,366 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58. after waiting 0 ms 2023-08-18 03:02:23,366 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58. 2023-08-18 03:02:23,366 INFO [RS:0;jenkins-hbase4:38805] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-18 03:02:23,366 INFO [RS:0;jenkins-hbase4:38805] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-18 03:02:23,367 INFO [RS:0;jenkins-hbase4:38805] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-18 03:02:23,367 INFO [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-18 03:02:23,367 INFO [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-08-18 03:02:23,367 DEBUG [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(1478): Online Regions={6b5011f83909e605795a74cdf612ac58=hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58., 1ff6f8076f79980eeb8f20457c47fb98=TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98., 1588230740=hbase:meta,,1.1588230740, a9284c12aea305493291bf3e413c6c3d=TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d.} 2023-08-18 03:02:23,367 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-18 03:02:23,367 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-18 03:02:23,368 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-18 03:02:23,368 DEBUG [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(1504): Waiting on 1588230740, 1ff6f8076f79980eeb8f20457c47fb98, 6b5011f83909e605795a74cdf612ac58, a9284c12aea305493291bf3e413c6c3d 2023-08-18 03:02:23,368 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-18 03:02:23,368 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-18 03:02:23,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/namespace/6b5011f83909e605795a74cdf612ac58/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-18 03:02:23,376 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/hbase/meta/1588230740/recovered.edits/27.seqid, newMaxSeqId=27, maxSeqId=1 2023-08-18 03:02:23,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58. 2023-08-18 03:02:23,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6b5011f83909e605795a74cdf612ac58: 2023-08-18 03:02:23,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1692327605642.6b5011f83909e605795a74cdf612ac58. 2023-08-18 03:02:23,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1ff6f8076f79980eeb8f20457c47fb98, disabling compactions & flushes 2023-08-18 03:02:23,377 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:02:23,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:02:23,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. after waiting 0 ms 2023-08-18 03:02:23,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:02:23,377 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-18 03:02:23,377 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-18 03:02:23,377 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/bce4132d5d5c4a2d83b66f4cd79f120a, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/bbac1b77edf94f4680b1f095ddde99a9, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/103360e757dc4024b3d5889f3bcc3a0f] to archive 2023-08-18 03:02:23,377 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-18 03:02:23,378 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-18 03:02:23,378 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-18 03:02:23,380 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/bce4132d5d5c4a2d83b66f4cd79f120a to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/bce4132d5d5c4a2d83b66f4cd79f120a 2023-08-18 03:02:23,381 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/bbac1b77edf94f4680b1f095ddde99a9 to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/bbac1b77edf94f4680b1f095ddde99a9 2023-08-18 03:02:23,382 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/103360e757dc4024b3d5889f3bcc3a0f to hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/archive/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/info/103360e757dc4024b3d5889f3bcc3a0f 2023-08-18 03:02:23,385 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/1ff6f8076f79980eeb8f20457c47fb98/recovered.edits/327.seqid, newMaxSeqId=327, maxSeqId=113 2023-08-18 03:02:23,386 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:02:23,387 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1ff6f8076f79980eeb8f20457c47fb98: 2023-08-18 03:02:23,387 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,row0064,1692327640504.1ff6f8076f79980eeb8f20457c47fb98. 2023-08-18 03:02:23,387 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a9284c12aea305493291bf3e413c6c3d, disabling compactions & flushes 2023-08-18 03:02:23,387 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d. 2023-08-18 03:02:23,387 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d. 2023-08-18 03:02:23,387 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d. after waiting 0 ms 2023-08-18 03:02:23,387 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d. 2023-08-18 03:02:23,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/data/default/TestLogRolling-testLogRolling/a9284c12aea305493291bf3e413c6c3d/recovered.edits/118.seqid, newMaxSeqId=118, maxSeqId=113 2023-08-18 03:02:23,390 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d. 2023-08-18 03:02:23,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a9284c12aea305493291bf3e413c6c3d: 2023-08-18 03:02:23,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,,1692327640504.a9284c12aea305493291bf3e413c6c3d. 2023-08-18 03:02:23,568 INFO [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38805,1692327604836; all regions closed. 2023-08-18 03:02:23,574 DEBUG [RS:0;jenkins-hbase4:38805] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/oldWALs 2023-08-18 03:02:23,574 INFO [RS:0;jenkins-hbase4:38805] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38805%2C1692327604836.meta:.meta(num 1692327605566) 2023-08-18 03:02:23,577 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/WALs/jenkins-hbase4.apache.org,38805,1692327604836/jenkins-hbase4.apache.org%2C38805%2C1692327604836.1692327743144 not finished, retry = 0 2023-08-18 03:02:23,680 DEBUG [RS:0;jenkins-hbase4:38805] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/oldWALs 2023-08-18 03:02:23,680 INFO [RS:0;jenkins-hbase4:38805] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38805%2C1692327604836:(num 1692327743144) 2023-08-18 03:02:23,680 DEBUG [RS:0;jenkins-hbase4:38805] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 03:02:23,680 INFO [RS:0;jenkins-hbase4:38805] regionserver.LeaseManager(133): Closed leases 2023-08-18 03:02:23,680 INFO [RS:0;jenkins-hbase4:38805] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-18 03:02:23,680 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-18 03:02:23,681 INFO [RS:0;jenkins-hbase4:38805] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38805 2023-08-18 03:02:23,683 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): regionserver:38805-0x1001d5529a90001, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38805,1692327604836 2023-08-18 03:02:23,683 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-18 03:02:23,683 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): regionserver:38805-0x1001d5529a90001, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-18 03:02:23,684 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38805,1692327604836] 2023-08-18 03:02:23,684 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38805,1692327604836; numProcessing=1 2023-08-18 03:02:23,686 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,38805,1692327604836 already deleted, retry=false 2023-08-18 03:02:23,687 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38805,1692327604836 expired; onlineServers=0 2023-08-18 03:02:23,687 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39179,1692327604776' ***** 2023-08-18 03:02:23,687 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-18 03:02:23,687 DEBUG [M:0;jenkins-hbase4:39179] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7f06aef4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-18 03:02:23,687 INFO [M:0;jenkins-hbase4:39179] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39179,1692327604776 2023-08-18 03:02:23,687 INFO [M:0;jenkins-hbase4:39179] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39179,1692327604776; all regions closed. 2023-08-18 03:02:23,687 DEBUG [M:0;jenkins-hbase4:39179] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 03:02:23,687 DEBUG [M:0;jenkins-hbase4:39179] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-18 03:02:23,687 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-18 03:02:23,687 DEBUG [M:0;jenkins-hbase4:39179] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-18 03:02:23,687 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692327605078] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692327605078,5,FailOnTimeoutGroup] 2023-08-18 03:02:23,688 INFO [M:0;jenkins-hbase4:39179] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-18 03:02:23,688 INFO [M:0;jenkins-hbase4:39179] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-18 03:02:23,687 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692327605077] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692327605077,5,FailOnTimeoutGroup] 2023-08-18 03:02:23,688 INFO [M:0;jenkins-hbase4:39179] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-18 03:02:23,688 DEBUG [M:0;jenkins-hbase4:39179] master.HMaster(1512): Stopping service threads 2023-08-18 03:02:23,688 INFO [M:0;jenkins-hbase4:39179] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-18 03:02:23,688 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-18 03:02:23,688 ERROR [M:0;jenkins-hbase4:39179] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-18 03:02:23,689 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:02:23,689 INFO [M:0;jenkins-hbase4:39179] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-18 03:02:23,689 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-18 03:02:23,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-18 03:02:23,689 DEBUG [M:0;jenkins-hbase4:39179] zookeeper.ZKUtil(398): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-18 03:02:23,689 WARN [M:0;jenkins-hbase4:39179] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-18 03:02:23,689 INFO [M:0;jenkins-hbase4:39179] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-18 03:02:23,689 INFO [M:0;jenkins-hbase4:39179] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-18 03:02:23,690 DEBUG [M:0;jenkins-hbase4:39179] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-18 03:02:23,690 INFO [M:0;jenkins-hbase4:39179] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:02:23,690 DEBUG [M:0;jenkins-hbase4:39179] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:02:23,690 DEBUG [M:0;jenkins-hbase4:39179] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-18 03:02:23,690 DEBUG [M:0;jenkins-hbase4:39179] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:02:23,690 INFO [M:0;jenkins-hbase4:39179] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=64.69 KB heapSize=78.42 KB 2023-08-18 03:02:23,701 INFO [M:0;jenkins-hbase4:39179] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=64.69 KB at sequenceid=160 (bloomFilter=true), to=hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/80c4cca55ad2413ba5a3714f30dd8ff6 2023-08-18 03:02:23,707 INFO [M:0;jenkins-hbase4:39179] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 80c4cca55ad2413ba5a3714f30dd8ff6 2023-08-18 03:02:23,708 DEBUG [M:0;jenkins-hbase4:39179] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/80c4cca55ad2413ba5a3714f30dd8ff6 as hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/80c4cca55ad2413ba5a3714f30dd8ff6 2023-08-18 03:02:23,712 INFO [M:0;jenkins-hbase4:39179] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 80c4cca55ad2413ba5a3714f30dd8ff6 2023-08-18 03:02:23,712 INFO [M:0;jenkins-hbase4:39179] regionserver.HStore(1080): Added hdfs://localhost:45289/user/jenkins/test-data/c8382975-4ce6-2dd8-0a64-effa44c40fe3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/80c4cca55ad2413ba5a3714f30dd8ff6, entries=18, sequenceid=160, filesize=6.9 K 2023-08-18 03:02:23,713 INFO [M:0;jenkins-hbase4:39179] regionserver.HRegion(2948): Finished flush of dataSize ~64.69 KB/66244, heapSize ~78.41 KB/80288, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 23ms, sequenceid=160, compaction requested=false 2023-08-18 03:02:23,715 INFO [M:0;jenkins-hbase4:39179] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:02:23,715 DEBUG [M:0;jenkins-hbase4:39179] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-18 03:02:23,718 INFO [M:0;jenkins-hbase4:39179] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-18 03:02:23,718 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-18 03:02:23,718 INFO [M:0;jenkins-hbase4:39179] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39179 2023-08-18 03:02:23,720 DEBUG [M:0;jenkins-hbase4:39179] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,39179,1692327604776 already deleted, retry=false 2023-08-18 03:02:23,785 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): regionserver:38805-0x1001d5529a90001, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 03:02:23,785 INFO [RS:0;jenkins-hbase4:38805] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38805,1692327604836; zookeeper connection closed. 2023-08-18 03:02:23,785 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): regionserver:38805-0x1001d5529a90001, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 03:02:23,786 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@a99e44] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@a99e44 2023-08-18 03:02:23,786 INFO [Listener at localhost/43431] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-18 03:02:23,885 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 03:02:23,885 INFO [M:0;jenkins-hbase4:39179] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39179,1692327604776; zookeeper connection closed. 2023-08-18 03:02:23,885 DEBUG [Listener at localhost/43431-EventThread] zookeeper.ZKWatcher(600): master:39179-0x1001d5529a90000, quorum=127.0.0.1:55029, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 03:02:23,886 WARN [Listener at localhost/43431] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-18 03:02:23,890 INFO [Listener at localhost/43431] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 03:02:23,995 WARN [BP-181493628-172.31.14.131-1692327603903 heartbeating to localhost/127.0.0.1:45289] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-18 03:02:23,995 WARN [BP-181493628-172.31.14.131-1692327603903 heartbeating to localhost/127.0.0.1:45289] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-181493628-172.31.14.131-1692327603903 (Datanode Uuid 22b65a7e-62c2-431d-b735-4f305df9d260) service to localhost/127.0.0.1:45289 2023-08-18 03:02:23,995 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/cluster_dd64d8b1-ed37-efa0-963c-da8e38bf2d1b/dfs/data/data3/current/BP-181493628-172.31.14.131-1692327603903] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 03:02:23,996 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/cluster_dd64d8b1-ed37-efa0-963c-da8e38bf2d1b/dfs/data/data4/current/BP-181493628-172.31.14.131-1692327603903] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 03:02:23,997 WARN [Listener at localhost/43431] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-18 03:02:24,000 INFO [Listener at localhost/43431] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 03:02:24,105 WARN [BP-181493628-172.31.14.131-1692327603903 heartbeating to localhost/127.0.0.1:45289] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-18 03:02:24,105 WARN [BP-181493628-172.31.14.131-1692327603903 heartbeating to localhost/127.0.0.1:45289] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-181493628-172.31.14.131-1692327603903 (Datanode Uuid caf26e79-78a9-46ea-82cd-efdce0fac523) service to localhost/127.0.0.1:45289 2023-08-18 03:02:24,105 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/cluster_dd64d8b1-ed37-efa0-963c-da8e38bf2d1b/dfs/data/data1/current/BP-181493628-172.31.14.131-1692327603903] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 03:02:24,106 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/cluster_dd64d8b1-ed37-efa0-963c-da8e38bf2d1b/dfs/data/data2/current/BP-181493628-172.31.14.131-1692327603903] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 03:02:24,116 INFO [Listener at localhost/43431] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 03:02:24,233 INFO [Listener at localhost/43431] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-18 03:02:24,275 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-18 03:02:24,285 INFO [Listener at localhost/43431] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=75 (was 65) - Thread LEAK? -, OpenFileDescriptor=459 (was 436) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=104 (was 99) - SystemLoadAverage LEAK? -, ProcessCount=172 (was 175), AvailableMemoryMB=1881 (was 2243) 2023-08-18 03:02:24,293 INFO [Listener at localhost/43431] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=76, OpenFileDescriptor=459, MaxFileDescriptor=60000, SystemLoadAverage=104, ProcessCount=172, AvailableMemoryMB=1881 2023-08-18 03:02:24,294 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-18 03:02:24,294 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/hadoop.log.dir so I do NOT create it in target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e 2023-08-18 03:02:24,294 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f6e1fd13-8de6-eee4-d07e-dbafcc02ff10/hadoop.tmp.dir so I do NOT create it in target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e 2023-08-18 03:02:24,294 INFO [Listener at localhost/43431] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/cluster_e68948b9-c4d2-07ea-5009-26cb263e42b5, deleteOnExit=true 2023-08-18 03:02:24,294 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-18 03:02:24,294 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/test.cache.data in system properties and HBase conf 2023-08-18 03:02:24,294 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/hadoop.tmp.dir in system properties and HBase conf 2023-08-18 03:02:24,295 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/hadoop.log.dir in system properties and HBase conf 2023-08-18 03:02:24,295 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-18 03:02:24,295 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-18 03:02:24,295 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-18 03:02:24,295 DEBUG [Listener at localhost/43431] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-18 03:02:24,295 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-18 03:02:24,295 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-18 03:02:24,295 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-18 03:02:24,295 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-18 03:02:24,296 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-18 03:02:24,296 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-18 03:02:24,296 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-18 03:02:24,296 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-18 03:02:24,296 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-18 03:02:24,296 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/nfs.dump.dir in system properties and HBase conf 2023-08-18 03:02:24,296 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/java.io.tmpdir in system properties and HBase conf 2023-08-18 03:02:24,296 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-18 03:02:24,296 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-18 03:02:24,297 INFO [Listener at localhost/43431] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-18 03:02:24,302 WARN [Listener at localhost/43431] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-18 03:02:24,302 WARN [Listener at localhost/43431] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-18 03:02:24,343 WARN [Listener at localhost/43431] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 03:02:24,344 INFO [Listener at localhost/43431] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 03:02:24,349 INFO [Listener at localhost/43431] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/java.io.tmpdir/Jetty_localhost_35383_hdfs____.2oruyy/webapp 2023-08-18 03:02:24,442 INFO [Listener at localhost/43431] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35383 2023-08-18 03:02:24,447 WARN [Listener at localhost/43431] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-18 03:02:24,447 WARN [Listener at localhost/43431] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-18 03:02:24,489 WARN [Listener at localhost/46109] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 03:02:24,499 WARN [Listener at localhost/46109] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-18 03:02:24,502 WARN [Listener at localhost/46109] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 03:02:24,503 INFO [Listener at localhost/46109] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 03:02:24,507 INFO [Listener at localhost/46109] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/java.io.tmpdir/Jetty_localhost_46293_datanode____.6fiscv/webapp 2023-08-18 03:02:24,599 INFO [Listener at localhost/46109] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46293 2023-08-18 03:02:24,605 WARN [Listener at localhost/33321] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 03:02:24,619 WARN [Listener at localhost/33321] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-18 03:02:24,621 WARN [Listener at localhost/33321] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-18 03:02:24,622 INFO [Listener at localhost/33321] log.Slf4jLog(67): jetty-6.1.26 2023-08-18 03:02:24,626 INFO [Listener at localhost/33321] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/java.io.tmpdir/Jetty_localhost_45401_datanode____.58c2kl/webapp 2023-08-18 03:02:24,698 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc376f3a5f67069bd: Processing first storage report for DS-0cbfa791-5769-425d-922f-2e249c6511b7 from datanode af1d0b66-1666-4ab0-862c-b086f5042729 2023-08-18 03:02:24,698 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc376f3a5f67069bd: from storage DS-0cbfa791-5769-425d-922f-2e249c6511b7 node DatanodeRegistration(127.0.0.1:38849, datanodeUuid=af1d0b66-1666-4ab0-862c-b086f5042729, infoPort=43193, infoSecurePort=0, ipcPort=33321, storageInfo=lv=-57;cid=testClusterID;nsid=1539655340;c=1692327744304), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 03:02:24,698 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc376f3a5f67069bd: Processing first storage report for DS-9dffa30b-d4b2-4fdf-a648-82c2abe5e0bb from datanode af1d0b66-1666-4ab0-862c-b086f5042729 2023-08-18 03:02:24,698 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc376f3a5f67069bd: from storage DS-9dffa30b-d4b2-4fdf-a648-82c2abe5e0bb node DatanodeRegistration(127.0.0.1:38849, datanodeUuid=af1d0b66-1666-4ab0-862c-b086f5042729, infoPort=43193, infoSecurePort=0, ipcPort=33321, storageInfo=lv=-57;cid=testClusterID;nsid=1539655340;c=1692327744304), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 03:02:24,720 INFO [Listener at localhost/33321] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45401 2023-08-18 03:02:24,763 WARN [Listener at localhost/45731] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-18 03:02:24,860 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd6f9c0a030329c0d: Processing first storage report for DS-60a80ef4-9e29-48b5-993d-d0a70227304f from datanode 8aeace5f-70c3-449b-abab-a19c18fc132b 2023-08-18 03:02:24,860 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd6f9c0a030329c0d: from storage DS-60a80ef4-9e29-48b5-993d-d0a70227304f node DatanodeRegistration(127.0.0.1:34323, datanodeUuid=8aeace5f-70c3-449b-abab-a19c18fc132b, infoPort=42787, infoSecurePort=0, ipcPort=45731, storageInfo=lv=-57;cid=testClusterID;nsid=1539655340;c=1692327744304), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-18 03:02:24,860 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd6f9c0a030329c0d: Processing first storage report for DS-cfcaecd6-ea5b-4f6a-a080-b567ff56090e from datanode 8aeace5f-70c3-449b-abab-a19c18fc132b 2023-08-18 03:02:24,860 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd6f9c0a030329c0d: from storage DS-cfcaecd6-ea5b-4f6a-a080-b567ff56090e node DatanodeRegistration(127.0.0.1:34323, datanodeUuid=8aeace5f-70c3-449b-abab-a19c18fc132b, infoPort=42787, infoSecurePort=0, ipcPort=45731, storageInfo=lv=-57;cid=testClusterID;nsid=1539655340;c=1692327744304), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-18 03:02:24,869 DEBUG [Listener at localhost/45731] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e 2023-08-18 03:02:24,871 INFO [Listener at localhost/45731] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/cluster_e68948b9-c4d2-07ea-5009-26cb263e42b5/zookeeper_0, clientPort=49355, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/cluster_e68948b9-c4d2-07ea-5009-26cb263e42b5/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/cluster_e68948b9-c4d2-07ea-5009-26cb263e42b5/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-18 03:02:24,872 INFO [Listener at localhost/45731] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=49355 2023-08-18 03:02:24,873 INFO [Listener at localhost/45731] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:02:24,873 INFO [Listener at localhost/45731] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:02:24,889 INFO [Listener at localhost/45731] util.FSUtils(471): Created version file at hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f with version=8 2023-08-18 03:02:24,889 INFO [Listener at localhost/45731] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:44229/user/jenkins/test-data/cdf1f99e-8d5b-0dcb-f178-7174d58df101/hbase-staging 2023-08-18 03:02:24,891 INFO [Listener at localhost/45731] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-18 03:02:24,892 INFO [Listener at localhost/45731] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 03:02:24,892 INFO [Listener at localhost/45731] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-18 03:02:24,892 INFO [Listener at localhost/45731] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-18 03:02:24,892 INFO [Listener at localhost/45731] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 03:02:24,892 INFO [Listener at localhost/45731] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-18 03:02:24,892 INFO [Listener at localhost/45731] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-18 03:02:24,893 INFO [Listener at localhost/45731] ipc.NettyRpcServer(120): Bind to /172.31.14.131:32959 2023-08-18 03:02:24,894 INFO [Listener at localhost/45731] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:02:24,895 INFO [Listener at localhost/45731] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:02:24,896 INFO [Listener at localhost/45731] zookeeper.RecoverableZooKeeper(93): Process identifier=master:32959 connecting to ZooKeeper ensemble=127.0.0.1:49355 2023-08-18 03:02:24,908 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:329590x0, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-18 03:02:24,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:32959-0x1001d574d0d0000 connected 2023-08-18 03:02:24,925 DEBUG [Listener at localhost/45731] zookeeper.ZKUtil(164): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-18 03:02:24,926 DEBUG [Listener at localhost/45731] zookeeper.ZKUtil(164): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 03:02:24,926 DEBUG [Listener at localhost/45731] zookeeper.ZKUtil(164): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-18 03:02:24,926 DEBUG [Listener at localhost/45731] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=32959 2023-08-18 03:02:24,927 DEBUG [Listener at localhost/45731] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=32959 2023-08-18 03:02:24,927 DEBUG [Listener at localhost/45731] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=32959 2023-08-18 03:02:24,927 DEBUG [Listener at localhost/45731] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=32959 2023-08-18 03:02:24,927 DEBUG [Listener at localhost/45731] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=32959 2023-08-18 03:02:24,927 INFO [Listener at localhost/45731] master.HMaster(444): hbase.rootdir=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f, hbase.cluster.distributed=false 2023-08-18 03:02:24,940 INFO [Listener at localhost/45731] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-18 03:02:24,940 INFO [Listener at localhost/45731] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 03:02:24,940 INFO [Listener at localhost/45731] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-18 03:02:24,940 INFO [Listener at localhost/45731] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-18 03:02:24,940 INFO [Listener at localhost/45731] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-18 03:02:24,940 INFO [Listener at localhost/45731] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-18 03:02:24,940 INFO [Listener at localhost/45731] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-18 03:02:24,942 INFO [Listener at localhost/45731] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42853 2023-08-18 03:02:24,943 INFO [Listener at localhost/45731] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-18 03:02:24,943 DEBUG [Listener at localhost/45731] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-18 03:02:24,944 INFO [Listener at localhost/45731] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:02:24,945 INFO [Listener at localhost/45731] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:02:24,945 INFO [Listener at localhost/45731] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42853 connecting to ZooKeeper ensemble=127.0.0.1:49355 2023-08-18 03:02:24,948 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): regionserver:428530x0, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-18 03:02:24,950 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42853-0x1001d574d0d0001 connected 2023-08-18 03:02:24,950 DEBUG [Listener at localhost/45731] zookeeper.ZKUtil(164): regionserver:42853-0x1001d574d0d0001, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-18 03:02:24,950 DEBUG [Listener at localhost/45731] zookeeper.ZKUtil(164): regionserver:42853-0x1001d574d0d0001, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 03:02:24,951 DEBUG [Listener at localhost/45731] zookeeper.ZKUtil(164): regionserver:42853-0x1001d574d0d0001, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-18 03:02:24,951 DEBUG [Listener at localhost/45731] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42853 2023-08-18 03:02:24,951 DEBUG [Listener at localhost/45731] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42853 2023-08-18 03:02:24,952 DEBUG [Listener at localhost/45731] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42853 2023-08-18 03:02:24,952 DEBUG [Listener at localhost/45731] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42853 2023-08-18 03:02:24,953 DEBUG [Listener at localhost/45731] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42853 2023-08-18 03:02:24,954 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,32959,1692327744891 2023-08-18 03:02:24,955 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-18 03:02:24,955 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,32959,1692327744891 2023-08-18 03:02:24,957 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-18 03:02:24,957 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): regionserver:42853-0x1001d574d0d0001, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-18 03:02:24,957 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:02:24,958 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-18 03:02:24,958 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,32959,1692327744891 from backup master directory 2023-08-18 03:02:24,958 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-18 03:02:24,962 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,32959,1692327744891 2023-08-18 03:02:24,962 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-18 03:02:24,962 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-18 03:02:24,962 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,32959,1692327744891 2023-08-18 03:02:24,974 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/hbase.id with ID: e27330ba-dd26-4e58-b320-8dad757f792a 2023-08-18 03:02:24,985 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:02:24,988 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:02:24,997 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x7995bd3f to 127.0.0.1:49355 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-18 03:02:25,001 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@274375f2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-18 03:02:25,001 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-18 03:02:25,001 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-18 03:02:25,001 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-18 03:02:25,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/MasterData/data/master/store-tmp 2023-08-18 03:02:25,014 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:02:25,015 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-18 03:02:25,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:02:25,015 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:02:25,015 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-18 03:02:25,015 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:02:25,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:02:25,015 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-18 03:02:25,015 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/MasterData/WALs/jenkins-hbase4.apache.org,32959,1692327744891 2023-08-18 03:02:25,018 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C32959%2C1692327744891, suffix=, logDir=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/MasterData/WALs/jenkins-hbase4.apache.org,32959,1692327744891, archiveDir=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/MasterData/oldWALs, maxLogs=10 2023-08-18 03:02:25,033 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34323,DS-60a80ef4-9e29-48b5-993d-d0a70227304f,DISK] 2023-08-18 03:02:25,033 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38849,DS-0cbfa791-5769-425d-922f-2e249c6511b7,DISK] 2023-08-18 03:02:25,036 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/MasterData/WALs/jenkins-hbase4.apache.org,32959,1692327744891/jenkins-hbase4.apache.org%2C32959%2C1692327744891.1692327745018 2023-08-18 03:02:25,037 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38849,DS-0cbfa791-5769-425d-922f-2e249c6511b7,DISK], DatanodeInfoWithStorage[127.0.0.1:34323,DS-60a80ef4-9e29-48b5-993d-d0a70227304f,DISK]] 2023-08-18 03:02:25,037 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-18 03:02:25,037 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:02:25,037 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-18 03:02:25,037 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-18 03:02:25,039 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-18 03:02:25,040 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-18 03:02:25,041 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-18 03:02:25,041 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:02:25,042 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-18 03:02:25,042 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-18 03:02:25,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-18 03:02:25,046 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-18 03:02:25,047 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=708161, jitterRate=-0.09952761232852936}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-18 03:02:25,047 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-18 03:02:25,047 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-18 03:02:25,048 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-18 03:02:25,048 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-18 03:02:25,048 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-18 03:02:25,048 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-18 03:02:25,049 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-08-18 03:02:25,049 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-18 03:02:25,049 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-18 03:02:25,050 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-18 03:02:25,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-18 03:02:25,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-18 03:02:25,061 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-18 03:02:25,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-18 03:02:25,061 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-18 03:02:25,063 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:02:25,064 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-18 03:02:25,064 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-18 03:02:25,065 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-18 03:02:25,067 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-18 03:02:25,067 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): regionserver:42853-0x1001d574d0d0001, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-18 03:02:25,067 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:02:25,068 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,32959,1692327744891, sessionid=0x1001d574d0d0000, setting cluster-up flag (Was=false) 2023-08-18 03:02:25,071 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:02:25,076 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-18 03:02:25,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,32959,1692327744891 2023-08-18 03:02:25,080 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:02:25,085 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-18 03:02:25,086 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,32959,1692327744891 2023-08-18 03:02:25,086 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/.hbase-snapshot/.tmp 2023-08-18 03:02:25,088 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-18 03:02:25,088 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 03:02:25,088 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 03:02:25,088 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 03:02:25,088 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-18 03:02:25,088 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-18 03:02:25,088 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:02:25,088 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-18 03:02:25,088 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:02:25,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692327775090 2023-08-18 03:02:25,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-18 03:02:25,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-18 03:02:25,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-18 03:02:25,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-18 03:02:25,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-18 03:02:25,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-18 03:02:25,090 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-18 03:02:25,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-18 03:02:25,090 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-18 03:02:25,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-18 03:02:25,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-18 03:02:25,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-18 03:02:25,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-18 03:02:25,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-18 03:02:25,091 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692327745091,5,FailOnTimeoutGroup] 2023-08-18 03:02:25,091 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-18 03:02:25,092 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692327745091,5,FailOnTimeoutGroup] 2023-08-18 03:02:25,092 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-18 03:02:25,092 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-18 03:02:25,092 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-18 03:02:25,092 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-18 03:02:25,100 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-18 03:02:25,101 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-18 03:02:25,101 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f 2023-08-18 03:02:25,109 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:02:25,110 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-18 03:02:25,112 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/info 2023-08-18 03:02:25,112 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-18 03:02:25,112 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:02:25,113 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-18 03:02:25,114 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/rep_barrier 2023-08-18 03:02:25,114 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-18 03:02:25,114 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:02:25,114 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-18 03:02:25,115 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/table 2023-08-18 03:02:25,116 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-18 03:02:25,116 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:02:25,117 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740 2023-08-18 03:02:25,117 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740 2023-08-18 03:02:25,119 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-18 03:02:25,120 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-18 03:02:25,122 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-18 03:02:25,122 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=769212, jitterRate=-0.0218973308801651}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-18 03:02:25,122 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-18 03:02:25,122 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-18 03:02:25,122 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-18 03:02:25,122 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-18 03:02:25,122 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-18 03:02:25,122 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-18 03:02:25,122 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-18 03:02:25,123 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-18 03:02:25,123 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-18 03:02:25,123 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-18 03:02:25,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-18 03:02:25,125 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-18 03:02:25,126 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-18 03:02:25,154 INFO [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(951): ClusterId : e27330ba-dd26-4e58-b320-8dad757f792a 2023-08-18 03:02:25,155 DEBUG [RS:0;jenkins-hbase4:42853] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-18 03:02:25,157 DEBUG [RS:0;jenkins-hbase4:42853] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-18 03:02:25,157 DEBUG [RS:0;jenkins-hbase4:42853] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-18 03:02:25,160 DEBUG [RS:0;jenkins-hbase4:42853] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-18 03:02:25,160 DEBUG [RS:0;jenkins-hbase4:42853] zookeeper.ReadOnlyZKClient(139): Connect 0x1aef6b75 to 127.0.0.1:49355 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-18 03:02:25,164 DEBUG [RS:0;jenkins-hbase4:42853] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@14288658, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-18 03:02:25,164 DEBUG [RS:0;jenkins-hbase4:42853] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@10170005, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-18 03:02:25,175 DEBUG [RS:0;jenkins-hbase4:42853] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:42853 2023-08-18 03:02:25,175 INFO [RS:0;jenkins-hbase4:42853] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-18 03:02:25,175 INFO [RS:0;jenkins-hbase4:42853] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-18 03:02:25,175 DEBUG [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(1022): About to register with Master. 2023-08-18 03:02:25,175 INFO [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,32959,1692327744891 with isa=jenkins-hbase4.apache.org/172.31.14.131:42853, startcode=1692327744939 2023-08-18 03:02:25,176 DEBUG [RS:0;jenkins-hbase4:42853] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-18 03:02:25,178 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36559, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-08-18 03:02:25,179 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=32959] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42853,1692327744939 2023-08-18 03:02:25,179 DEBUG [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f 2023-08-18 03:02:25,179 DEBUG [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:46109 2023-08-18 03:02:25,179 DEBUG [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-18 03:02:25,180 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-18 03:02:25,181 DEBUG [RS:0;jenkins-hbase4:42853] zookeeper.ZKUtil(162): regionserver:42853-0x1001d574d0d0001, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42853,1692327744939 2023-08-18 03:02:25,181 WARN [RS:0;jenkins-hbase4:42853] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-18 03:02:25,181 INFO [RS:0;jenkins-hbase4:42853] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-18 03:02:25,181 DEBUG [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(1948): logDir=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/WALs/jenkins-hbase4.apache.org,42853,1692327744939 2023-08-18 03:02:25,181 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42853,1692327744939] 2023-08-18 03:02:25,184 DEBUG [RS:0;jenkins-hbase4:42853] zookeeper.ZKUtil(162): regionserver:42853-0x1001d574d0d0001, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42853,1692327744939 2023-08-18 03:02:25,185 DEBUG [RS:0;jenkins-hbase4:42853] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-18 03:02:25,185 INFO [RS:0;jenkins-hbase4:42853] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-18 03:02:25,186 INFO [RS:0;jenkins-hbase4:42853] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-18 03:02:25,186 INFO [RS:0;jenkins-hbase4:42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-18 03:02:25,187 INFO [RS:0;jenkins-hbase4:42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-18 03:02:25,187 INFO [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-18 03:02:25,188 INFO [RS:0;jenkins-hbase4:42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-18 03:02:25,188 DEBUG [RS:0;jenkins-hbase4:42853] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:02:25,188 DEBUG [RS:0;jenkins-hbase4:42853] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:02:25,188 DEBUG [RS:0;jenkins-hbase4:42853] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:02:25,188 DEBUG [RS:0;jenkins-hbase4:42853] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:02:25,188 DEBUG [RS:0;jenkins-hbase4:42853] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:02:25,188 DEBUG [RS:0;jenkins-hbase4:42853] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-18 03:02:25,188 DEBUG [RS:0;jenkins-hbase4:42853] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:02:25,188 DEBUG [RS:0;jenkins-hbase4:42853] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:02:25,188 DEBUG [RS:0;jenkins-hbase4:42853] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:02:25,188 DEBUG [RS:0;jenkins-hbase4:42853] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-18 03:02:25,189 INFO [RS:0;jenkins-hbase4:42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-18 03:02:25,189 INFO [RS:0;jenkins-hbase4:42853] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-18 03:02:25,189 INFO [RS:0;jenkins-hbase4:42853] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-18 03:02:25,200 INFO [RS:0;jenkins-hbase4:42853] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-18 03:02:25,200 INFO [RS:0;jenkins-hbase4:42853] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42853,1692327744939-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-18 03:02:25,206 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-18 03:02:25,211 INFO [RS:0;jenkins-hbase4:42853] regionserver.Replication(203): jenkins-hbase4.apache.org,42853,1692327744939 started 2023-08-18 03:02:25,211 INFO [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,42853,1692327744939, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42853, sessionid=0x1001d574d0d0001 2023-08-18 03:02:25,211 DEBUG [RS:0;jenkins-hbase4:42853] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-18 03:02:25,211 DEBUG [RS:0;jenkins-hbase4:42853] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42853,1692327744939 2023-08-18 03:02:25,211 DEBUG [RS:0;jenkins-hbase4:42853] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42853,1692327744939' 2023-08-18 03:02:25,212 DEBUG [RS:0;jenkins-hbase4:42853] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-18 03:02:25,212 DEBUG [RS:0;jenkins-hbase4:42853] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-18 03:02:25,212 DEBUG [RS:0;jenkins-hbase4:42853] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-18 03:02:25,212 DEBUG [RS:0;jenkins-hbase4:42853] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-18 03:02:25,212 DEBUG [RS:0;jenkins-hbase4:42853] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42853,1692327744939 2023-08-18 03:02:25,212 DEBUG [RS:0;jenkins-hbase4:42853] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42853,1692327744939' 2023-08-18 03:02:25,212 DEBUG [RS:0;jenkins-hbase4:42853] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-18 03:02:25,212 DEBUG [RS:0;jenkins-hbase4:42853] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-18 03:02:25,213 DEBUG [RS:0;jenkins-hbase4:42853] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-18 03:02:25,213 INFO [RS:0;jenkins-hbase4:42853] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-18 03:02:25,213 INFO [RS:0;jenkins-hbase4:42853] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-18 03:02:25,276 DEBUG [jenkins-hbase4:32959] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-18 03:02:25,277 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42853,1692327744939, state=OPENING 2023-08-18 03:02:25,278 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-18 03:02:25,281 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:02:25,281 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-18 03:02:25,281 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42853,1692327744939}] 2023-08-18 03:02:25,315 INFO [RS:0;jenkins-hbase4:42853] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42853%2C1692327744939, suffix=, logDir=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/WALs/jenkins-hbase4.apache.org,42853,1692327744939, archiveDir=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/oldWALs, maxLogs=32 2023-08-18 03:02:25,333 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38849,DS-0cbfa791-5769-425d-922f-2e249c6511b7,DISK] 2023-08-18 03:02:25,333 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34323,DS-60a80ef4-9e29-48b5-993d-d0a70227304f,DISK] 2023-08-18 03:02:25,335 INFO [RS:0;jenkins-hbase4:42853] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/WALs/jenkins-hbase4.apache.org,42853,1692327744939/jenkins-hbase4.apache.org%2C42853%2C1692327744939.1692327745315 2023-08-18 03:02:25,335 DEBUG [RS:0;jenkins-hbase4:42853] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38849,DS-0cbfa791-5769-425d-922f-2e249c6511b7,DISK], DatanodeInfoWithStorage[127.0.0.1:34323,DS-60a80ef4-9e29-48b5-993d-d0a70227304f,DISK]] 2023-08-18 03:02:25,435 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42853,1692327744939 2023-08-18 03:02:25,435 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-18 03:02:25,437 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55588, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-18 03:02:25,441 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-18 03:02:25,441 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-18 03:02:25,442 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42853%2C1692327744939.meta, suffix=.meta, logDir=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/WALs/jenkins-hbase4.apache.org,42853,1692327744939, archiveDir=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/oldWALs, maxLogs=32 2023-08-18 03:02:25,458 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34323,DS-60a80ef4-9e29-48b5-993d-d0a70227304f,DISK] 2023-08-18 03:02:25,459 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38849,DS-0cbfa791-5769-425d-922f-2e249c6511b7,DISK] 2023-08-18 03:02:25,460 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/WALs/jenkins-hbase4.apache.org,42853,1692327744939/jenkins-hbase4.apache.org%2C42853%2C1692327744939.meta.1692327745443.meta 2023-08-18 03:02:25,461 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34323,DS-60a80ef4-9e29-48b5-993d-d0a70227304f,DISK], DatanodeInfoWithStorage[127.0.0.1:38849,DS-0cbfa791-5769-425d-922f-2e249c6511b7,DISK]] 2023-08-18 03:02:25,461 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-18 03:02:25,461 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-18 03:02:25,461 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-18 03:02:25,461 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-18 03:02:25,461 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-18 03:02:25,461 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:02:25,461 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-18 03:02:25,461 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-18 03:02:25,463 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-18 03:02:25,464 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/info 2023-08-18 03:02:25,464 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/info 2023-08-18 03:02:25,464 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-18 03:02:25,464 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:02:25,464 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-18 03:02:25,465 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/rep_barrier 2023-08-18 03:02:25,465 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/rep_barrier 2023-08-18 03:02:25,466 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-18 03:02:25,466 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:02:25,466 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-18 03:02:25,467 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/table 2023-08-18 03:02:25,467 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/table 2023-08-18 03:02:25,467 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-18 03:02:25,468 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:02:25,468 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740 2023-08-18 03:02:25,470 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740 2023-08-18 03:02:25,471 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-18 03:02:25,473 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-18 03:02:25,473 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=811498, jitterRate=0.031873270869255066}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-18 03:02:25,473 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-18 03:02:25,474 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1692327745435 2023-08-18 03:02:25,478 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-18 03:02:25,478 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-18 03:02:25,478 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42853,1692327744939, state=OPEN 2023-08-18 03:02:25,480 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-18 03:02:25,480 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-18 03:02:25,482 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-18 03:02:25,482 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42853,1692327744939 in 199 msec 2023-08-18 03:02:25,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-18 03:02:25,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 359 msec 2023-08-18 03:02:25,484 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 396 msec 2023-08-18 03:02:25,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1692327745484, completionTime=-1 2023-08-18 03:02:25,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-18 03:02:25,485 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-18 03:02:25,487 DEBUG [hconnection-0x14b7bcd6-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-18 03:02:25,489 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55592, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-18 03:02:25,490 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-18 03:02:25,490 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692327805490 2023-08-18 03:02:25,490 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692327865490 2023-08-18 03:02:25,490 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-18 03:02:25,496 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,32959,1692327744891-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-18 03:02:25,496 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,32959,1692327744891-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-18 03:02:25,496 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,32959,1692327744891-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-18 03:02:25,496 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:32959, period=300000, unit=MILLISECONDS is enabled. 2023-08-18 03:02:25,496 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-18 03:02:25,496 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-18 03:02:25,496 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-18 03:02:25,497 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-18 03:02:25,497 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-18 03:02:25,498 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-18 03:02:25,499 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-18 03:02:25,500 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/.tmp/data/hbase/namespace/50dd250c2e4b65fec957774a1de7b952 2023-08-18 03:02:25,501 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/.tmp/data/hbase/namespace/50dd250c2e4b65fec957774a1de7b952 empty. 2023-08-18 03:02:25,501 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/.tmp/data/hbase/namespace/50dd250c2e4b65fec957774a1de7b952 2023-08-18 03:02:25,501 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-18 03:02:25,511 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-18 03:02:25,512 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 50dd250c2e4b65fec957774a1de7b952, NAME => 'hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/.tmp 2023-08-18 03:02:25,519 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:02:25,519 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 50dd250c2e4b65fec957774a1de7b952, disabling compactions & flushes 2023-08-18 03:02:25,519 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952. 2023-08-18 03:02:25,520 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952. 2023-08-18 03:02:25,520 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952. after waiting 0 ms 2023-08-18 03:02:25,520 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952. 2023-08-18 03:02:25,520 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952. 2023-08-18 03:02:25,520 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 50dd250c2e4b65fec957774a1de7b952: 2023-08-18 03:02:25,522 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-18 03:02:25,523 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692327745523"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692327745523"}]},"ts":"1692327745523"} 2023-08-18 03:02:25,524 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-18 03:02:25,525 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-18 03:02:25,525 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692327745525"}]},"ts":"1692327745525"} 2023-08-18 03:02:25,526 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-18 03:02:25,533 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=50dd250c2e4b65fec957774a1de7b952, ASSIGN}] 2023-08-18 03:02:25,534 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=50dd250c2e4b65fec957774a1de7b952, ASSIGN 2023-08-18 03:02:25,535 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=50dd250c2e4b65fec957774a1de7b952, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42853,1692327744939; forceNewPlan=false, retain=false 2023-08-18 03:02:25,686 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=50dd250c2e4b65fec957774a1de7b952, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42853,1692327744939 2023-08-18 03:02:25,686 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692327745686"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692327745686"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692327745686"}]},"ts":"1692327745686"} 2023-08-18 03:02:25,688 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 50dd250c2e4b65fec957774a1de7b952, server=jenkins-hbase4.apache.org,42853,1692327744939}] 2023-08-18 03:02:25,843 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952. 2023-08-18 03:02:25,843 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 50dd250c2e4b65fec957774a1de7b952, NAME => 'hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952.', STARTKEY => '', ENDKEY => ''} 2023-08-18 03:02:25,843 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 50dd250c2e4b65fec957774a1de7b952 2023-08-18 03:02:25,843 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-18 03:02:25,843 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 50dd250c2e4b65fec957774a1de7b952 2023-08-18 03:02:25,843 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 50dd250c2e4b65fec957774a1de7b952 2023-08-18 03:02:25,844 INFO [StoreOpener-50dd250c2e4b65fec957774a1de7b952-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 50dd250c2e4b65fec957774a1de7b952 2023-08-18 03:02:25,846 DEBUG [StoreOpener-50dd250c2e4b65fec957774a1de7b952-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/namespace/50dd250c2e4b65fec957774a1de7b952/info 2023-08-18 03:02:25,846 DEBUG [StoreOpener-50dd250c2e4b65fec957774a1de7b952-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/namespace/50dd250c2e4b65fec957774a1de7b952/info 2023-08-18 03:02:25,846 INFO [StoreOpener-50dd250c2e4b65fec957774a1de7b952-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 50dd250c2e4b65fec957774a1de7b952 columnFamilyName info 2023-08-18 03:02:25,847 INFO [StoreOpener-50dd250c2e4b65fec957774a1de7b952-1] regionserver.HStore(310): Store=50dd250c2e4b65fec957774a1de7b952/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-18 03:02:25,847 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/namespace/50dd250c2e4b65fec957774a1de7b952 2023-08-18 03:02:25,848 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/namespace/50dd250c2e4b65fec957774a1de7b952 2023-08-18 03:02:25,851 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 50dd250c2e4b65fec957774a1de7b952 2023-08-18 03:02:25,853 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/namespace/50dd250c2e4b65fec957774a1de7b952/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-18 03:02:25,853 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 50dd250c2e4b65fec957774a1de7b952; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=759397, jitterRate=-0.0343768447637558}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-18 03:02:25,853 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 50dd250c2e4b65fec957774a1de7b952: 2023-08-18 03:02:25,854 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952., pid=6, masterSystemTime=1692327745839 2023-08-18 03:02:25,855 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952. 2023-08-18 03:02:25,855 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952. 2023-08-18 03:02:25,856 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=50dd250c2e4b65fec957774a1de7b952, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42853,1692327744939 2023-08-18 03:02:25,856 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692327745856"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692327745856"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692327745856"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692327745856"}]},"ts":"1692327745856"} 2023-08-18 03:02:25,859 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-18 03:02:25,859 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 50dd250c2e4b65fec957774a1de7b952, server=jenkins-hbase4.apache.org,42853,1692327744939 in 169 msec 2023-08-18 03:02:25,860 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-18 03:02:25,861 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=50dd250c2e4b65fec957774a1de7b952, ASSIGN in 327 msec 2023-08-18 03:02:25,861 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-18 03:02:25,862 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692327745861"}]},"ts":"1692327745861"} 2023-08-18 03:02:25,863 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-18 03:02:25,865 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-18 03:02:25,866 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 369 msec 2023-08-18 03:02:25,898 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-18 03:02:25,900 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-18 03:02:25,900 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:02:25,903 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-18 03:02:25,910 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-18 03:02:25,913 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 9 msec 2023-08-18 03:02:25,915 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-18 03:02:25,920 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-18 03:02:25,924 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 9 msec 2023-08-18 03:02:25,928 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-18 03:02:25,931 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-18 03:02:25,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 0.969sec 2023-08-18 03:02:25,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-18 03:02:25,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-18 03:02:25,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-18 03:02:25,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,32959,1692327744891-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-18 03:02:25,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,32959,1692327744891-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-18 03:02:25,933 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-18 03:02:25,955 DEBUG [Listener at localhost/45731] zookeeper.ReadOnlyZKClient(139): Connect 0x7c29ecc4 to 127.0.0.1:49355 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-18 03:02:25,961 DEBUG [Listener at localhost/45731] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3db78391, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-18 03:02:25,963 DEBUG [hconnection-0x6e89727b-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-18 03:02:25,965 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55598, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-18 03:02:25,966 INFO [Listener at localhost/45731] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,32959,1692327744891 2023-08-18 03:02:25,966 INFO [Listener at localhost/45731] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-18 03:02:25,969 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-18 03:02:25,969 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:02:25,970 INFO [Listener at localhost/45731] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-18 03:02:25,970 INFO [Listener at localhost/45731] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-18 03:02:25,973 INFO [Listener at localhost/45731] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=test.com%2C8080%2C1, suffix=, logDir=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/WALs/test.com,8080,1, archiveDir=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/oldWALs, maxLogs=32 2023-08-18 03:02:25,994 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34323,DS-60a80ef4-9e29-48b5-993d-d0a70227304f,DISK] 2023-08-18 03:02:25,995 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38849,DS-0cbfa791-5769-425d-922f-2e249c6511b7,DISK] 2023-08-18 03:02:25,997 INFO [Listener at localhost/45731] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/WALs/test.com,8080,1/test.com%2C8080%2C1.1692327745973 2023-08-18 03:02:25,997 DEBUG [Listener at localhost/45731] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34323,DS-60a80ef4-9e29-48b5-993d-d0a70227304f,DISK], DatanodeInfoWithStorage[127.0.0.1:38849,DS-0cbfa791-5769-425d-922f-2e249c6511b7,DISK]] 2023-08-18 03:02:26,013 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34323,DS-60a80ef4-9e29-48b5-993d-d0a70227304f,DISK] 2023-08-18 03:02:26,014 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38849,DS-0cbfa791-5769-425d-922f-2e249c6511b7,DISK] 2023-08-18 03:02:26,015 INFO [Listener at localhost/45731] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/WALs/test.com,8080,1/test.com%2C8080%2C1.1692327745973 with entries=0, filesize=83 B; new WAL /user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/WALs/test.com,8080,1/test.com%2C8080%2C1.1692327745997 2023-08-18 03:02:26,015 DEBUG [Listener at localhost/45731] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34323,DS-60a80ef4-9e29-48b5-993d-d0a70227304f,DISK], DatanodeInfoWithStorage[127.0.0.1:38849,DS-0cbfa791-5769-425d-922f-2e249c6511b7,DISK]] 2023-08-18 03:02:26,015 DEBUG [Listener at localhost/45731] wal.AbstractFSWAL(716): hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/WALs/test.com,8080,1/test.com%2C8080%2C1.1692327745973 is not closed yet, will try archiving it next time 2023-08-18 03:02:26,020 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/WALs/test.com,8080,1/test.com%2C8080%2C1.1692327745973 not finished, retry = 0 2023-08-18 03:02:26,021 WARN [Close-WAL-Writer-1] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/WALs/test.com,8080,1/test.com%2C8080%2C1.1692327745997 not finished, retry = 0 2023-08-18 03:02:26,122 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/WALs/test.com,8080,1/test.com%2C8080%2C1.1692327745973 to hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/oldWALs/test.com%2C8080%2C1.1692327745973 2023-08-18 03:02:26,124 DEBUG [Listener at localhost/45731] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/oldWALs 2023-08-18 03:02:26,125 INFO [Listener at localhost/45731] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL test.com%2C8080%2C1:(num 1692327745997) 2023-08-18 03:02:26,125 INFO [Listener at localhost/45731] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-18 03:02:26,125 DEBUG [Listener at localhost/45731] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7c29ecc4 to 127.0.0.1:49355 2023-08-18 03:02:26,125 DEBUG [Listener at localhost/45731] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 03:02:26,126 DEBUG [Listener at localhost/45731] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-18 03:02:26,126 DEBUG [Listener at localhost/45731] util.JVMClusterUtil(257): Found active master hash=30025458, stopped=false 2023-08-18 03:02:26,126 INFO [Listener at localhost/45731] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,32959,1692327744891 2023-08-18 03:02:26,129 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): regionserver:42853-0x1001d574d0d0001, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-18 03:02:26,129 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-18 03:02:26,129 INFO [Listener at localhost/45731] procedure2.ProcedureExecutor(629): Stopping 2023-08-18 03:02:26,129 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:02:26,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42853-0x1001d574d0d0001, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 03:02:26,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-18 03:02:26,131 DEBUG [Listener at localhost/45731] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7995bd3f to 127.0.0.1:49355 2023-08-18 03:02:26,131 DEBUG [Listener at localhost/45731] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 03:02:26,131 INFO [Listener at localhost/45731] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42853,1692327744939' ***** 2023-08-18 03:02:26,131 INFO [Listener at localhost/45731] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-18 03:02:26,131 INFO [RS:0;jenkins-hbase4:42853] regionserver.HeapMemoryManager(220): Stopping 2023-08-18 03:02:26,131 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-18 03:02:26,131 INFO [RS:0;jenkins-hbase4:42853] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-18 03:02:26,132 INFO [RS:0;jenkins-hbase4:42853] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-18 03:02:26,132 INFO [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(3305): Received CLOSE for 50dd250c2e4b65fec957774a1de7b952 2023-08-18 03:02:26,132 INFO [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42853,1692327744939 2023-08-18 03:02:26,132 DEBUG [RS:0;jenkins-hbase4:42853] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1aef6b75 to 127.0.0.1:49355 2023-08-18 03:02:26,133 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 50dd250c2e4b65fec957774a1de7b952, disabling compactions & flushes 2023-08-18 03:02:26,133 DEBUG [RS:0;jenkins-hbase4:42853] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 03:02:26,133 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952. 2023-08-18 03:02:26,133 INFO [RS:0;jenkins-hbase4:42853] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-18 03:02:26,133 INFO [RS:0;jenkins-hbase4:42853] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-18 03:02:26,133 INFO [RS:0;jenkins-hbase4:42853] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-18 03:02:26,133 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952. 2023-08-18 03:02:26,133 INFO [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-18 03:02:26,133 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952. after waiting 0 ms 2023-08-18 03:02:26,133 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952. 2023-08-18 03:02:26,133 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 50dd250c2e4b65fec957774a1de7b952 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-18 03:02:26,133 INFO [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-08-18 03:02:26,133 DEBUG [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(1478): Online Regions={50dd250c2e4b65fec957774a1de7b952=hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952., 1588230740=hbase:meta,,1.1588230740} 2023-08-18 03:02:26,134 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-18 03:02:26,134 DEBUG [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(1504): Waiting on 1588230740, 50dd250c2e4b65fec957774a1de7b952 2023-08-18 03:02:26,134 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-18 03:02:26,134 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-18 03:02:26,134 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-18 03:02:26,134 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-18 03:02:26,134 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-08-18 03:02:26,149 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/namespace/50dd250c2e4b65fec957774a1de7b952/.tmp/info/07e54db170844fa2a369e539e191d71d 2023-08-18 03:02:26,152 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/.tmp/info/2aedd1b4d51c4cba8d10d97b39d01216 2023-08-18 03:02:26,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/namespace/50dd250c2e4b65fec957774a1de7b952/.tmp/info/07e54db170844fa2a369e539e191d71d as hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/namespace/50dd250c2e4b65fec957774a1de7b952/info/07e54db170844fa2a369e539e191d71d 2023-08-18 03:02:26,168 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/namespace/50dd250c2e4b65fec957774a1de7b952/info/07e54db170844fa2a369e539e191d71d, entries=2, sequenceid=6, filesize=4.8 K 2023-08-18 03:02:26,171 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 50dd250c2e4b65fec957774a1de7b952 in 38ms, sequenceid=6, compaction requested=false 2023-08-18 03:02:26,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/namespace/50dd250c2e4b65fec957774a1de7b952/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-18 03:02:26,184 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952. 2023-08-18 03:02:26,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 50dd250c2e4b65fec957774a1de7b952: 2023-08-18 03:02:26,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1692327745496.50dd250c2e4b65fec957774a1de7b952. 2023-08-18 03:02:26,184 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/.tmp/table/ac1ee9e16afd49f5a8160b5127786ce3 2023-08-18 03:02:26,190 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/.tmp/info/2aedd1b4d51c4cba8d10d97b39d01216 as hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/info/2aedd1b4d51c4cba8d10d97b39d01216 2023-08-18 03:02:26,190 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-18 03:02:26,190 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-18 03:02:26,195 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/info/2aedd1b4d51c4cba8d10d97b39d01216, entries=10, sequenceid=9, filesize=5.9 K 2023-08-18 03:02:26,196 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/.tmp/table/ac1ee9e16afd49f5a8160b5127786ce3 as hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/table/ac1ee9e16afd49f5a8160b5127786ce3 2023-08-18 03:02:26,200 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/table/ac1ee9e16afd49f5a8160b5127786ce3, entries=2, sequenceid=9, filesize=4.7 K 2023-08-18 03:02:26,201 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 67ms, sequenceid=9, compaction requested=false 2023-08-18 03:02:26,208 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-08-18 03:02:26,208 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-18 03:02:26,208 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-18 03:02:26,208 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-18 03:02:26,208 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-18 03:02:26,334 INFO [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42853,1692327744939; all regions closed. 2023-08-18 03:02:26,339 DEBUG [RS:0;jenkins-hbase4:42853] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/oldWALs 2023-08-18 03:02:26,340 INFO [RS:0;jenkins-hbase4:42853] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42853%2C1692327744939.meta:.meta(num 1692327745443) 2023-08-18 03:02:26,347 DEBUG [RS:0;jenkins-hbase4:42853] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/oldWALs 2023-08-18 03:02:26,347 INFO [RS:0;jenkins-hbase4:42853] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42853%2C1692327744939:(num 1692327745315) 2023-08-18 03:02:26,347 DEBUG [RS:0;jenkins-hbase4:42853] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 03:02:26,347 INFO [RS:0;jenkins-hbase4:42853] regionserver.LeaseManager(133): Closed leases 2023-08-18 03:02:26,347 INFO [RS:0;jenkins-hbase4:42853] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-18 03:02:26,347 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-18 03:02:26,348 INFO [RS:0;jenkins-hbase4:42853] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42853 2023-08-18 03:02:26,351 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): regionserver:42853-0x1001d574d0d0001, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42853,1692327744939 2023-08-18 03:02:26,351 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-18 03:02:26,352 ERROR [Listener at localhost/45731-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@29894d3 rejected from java.util.concurrent.ThreadPoolExecutor@6a70058[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1374) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:603) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-08-18 03:02:26,352 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): regionserver:42853-0x1001d574d0d0001, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-18 03:02:26,353 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42853,1692327744939] 2023-08-18 03:02:26,353 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42853,1692327744939; numProcessing=1 2023-08-18 03:02:26,354 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42853,1692327744939 already deleted, retry=false 2023-08-18 03:02:26,354 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42853,1692327744939 expired; onlineServers=0 2023-08-18 03:02:26,354 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,32959,1692327744891' ***** 2023-08-18 03:02:26,354 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-18 03:02:26,355 DEBUG [M:0;jenkins-hbase4:32959] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5f5b213, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-18 03:02:26,355 INFO [M:0;jenkins-hbase4:32959] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,32959,1692327744891 2023-08-18 03:02:26,355 INFO [M:0;jenkins-hbase4:32959] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,32959,1692327744891; all regions closed. 2023-08-18 03:02:26,355 DEBUG [M:0;jenkins-hbase4:32959] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-18 03:02:26,355 DEBUG [M:0;jenkins-hbase4:32959] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-18 03:02:26,355 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-18 03:02:26,355 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692327745091] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692327745091,5,FailOnTimeoutGroup] 2023-08-18 03:02:26,355 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692327745091] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692327745091,5,FailOnTimeoutGroup] 2023-08-18 03:02:26,355 DEBUG [M:0;jenkins-hbase4:32959] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-18 03:02:26,356 INFO [M:0;jenkins-hbase4:32959] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-18 03:02:26,356 INFO [M:0;jenkins-hbase4:32959] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-18 03:02:26,356 INFO [M:0;jenkins-hbase4:32959] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-18 03:02:26,356 DEBUG [M:0;jenkins-hbase4:32959] master.HMaster(1512): Stopping service threads 2023-08-18 03:02:26,356 INFO [M:0;jenkins-hbase4:32959] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-18 03:02:26,357 ERROR [M:0;jenkins-hbase4:32959] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] 2023-08-18 03:02:26,357 INFO [M:0;jenkins-hbase4:32959] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-18 03:02:26,357 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-18 03:02:26,357 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-18 03:02:26,357 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-18 03:02:26,357 DEBUG [M:0;jenkins-hbase4:32959] zookeeper.ZKUtil(398): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-18 03:02:26,357 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-18 03:02:26,357 WARN [M:0;jenkins-hbase4:32959] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-18 03:02:26,357 INFO [M:0;jenkins-hbase4:32959] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-18 03:02:26,358 INFO [M:0;jenkins-hbase4:32959] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-18 03:02:26,358 DEBUG [M:0;jenkins-hbase4:32959] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-18 03:02:26,358 INFO [M:0;jenkins-hbase4:32959] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:02:26,358 DEBUG [M:0;jenkins-hbase4:32959] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:02:26,358 DEBUG [M:0;jenkins-hbase4:32959] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-18 03:02:26,358 DEBUG [M:0;jenkins-hbase4:32959] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:02:26,358 INFO [M:0;jenkins-hbase4:32959] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.07 KB heapSize=29.55 KB 2023-08-18 03:02:26,368 INFO [M:0;jenkins-hbase4:32959] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.07 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/51b4e7fdd0314d238149b294a9c2f81f 2023-08-18 03:02:26,373 DEBUG [M:0;jenkins-hbase4:32959] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/51b4e7fdd0314d238149b294a9c2f81f as hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/51b4e7fdd0314d238149b294a9c2f81f 2023-08-18 03:02:26,377 INFO [M:0;jenkins-hbase4:32959] regionserver.HStore(1080): Added hdfs://localhost:46109/user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/51b4e7fdd0314d238149b294a9c2f81f, entries=8, sequenceid=66, filesize=6.3 K 2023-08-18 03:02:26,378 INFO [M:0;jenkins-hbase4:32959] regionserver.HRegion(2948): Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=66, compaction requested=false 2023-08-18 03:02:26,379 INFO [M:0;jenkins-hbase4:32959] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-18 03:02:26,379 DEBUG [M:0;jenkins-hbase4:32959] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-18 03:02:26,382 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/8bda4f7f-322b-c79e-20bd-02922724ac9f/MasterData/WALs/jenkins-hbase4.apache.org,32959,1692327744891/jenkins-hbase4.apache.org%2C32959%2C1692327744891.1692327745018 not finished, retry = 0 2023-08-18 03:02:26,483 INFO [M:0;jenkins-hbase4:32959] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-18 03:02:26,483 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-18 03:02:26,484 INFO [M:0;jenkins-hbase4:32959] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:32959 2023-08-18 03:02:26,487 DEBUG [M:0;jenkins-hbase4:32959] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,32959,1692327744891 already deleted, retry=false 2023-08-18 03:02:26,530 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): regionserver:42853-0x1001d574d0d0001, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 03:02:26,530 INFO [RS:0;jenkins-hbase4:42853] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42853,1692327744939; zookeeper connection closed. 2023-08-18 03:02:26,530 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): regionserver:42853-0x1001d574d0d0001, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 03:02:26,531 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@73236b3b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@73236b3b 2023-08-18 03:02:26,531 INFO [Listener at localhost/45731] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-18 03:02:26,630 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 03:02:26,630 INFO [M:0;jenkins-hbase4:32959] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,32959,1692327744891; zookeeper connection closed. 2023-08-18 03:02:26,630 DEBUG [Listener at localhost/45731-EventThread] zookeeper.ZKWatcher(600): master:32959-0x1001d574d0d0000, quorum=127.0.0.1:49355, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-18 03:02:26,632 WARN [Listener at localhost/45731] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-18 03:02:26,638 INFO [Listener at localhost/45731] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 03:02:26,741 WARN [BP-221994088-172.31.14.131-1692327744304 heartbeating to localhost/127.0.0.1:46109] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-18 03:02:26,741 WARN [BP-221994088-172.31.14.131-1692327744304 heartbeating to localhost/127.0.0.1:46109] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-221994088-172.31.14.131-1692327744304 (Datanode Uuid 8aeace5f-70c3-449b-abab-a19c18fc132b) service to localhost/127.0.0.1:46109 2023-08-18 03:02:26,742 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/cluster_e68948b9-c4d2-07ea-5009-26cb263e42b5/dfs/data/data3/current/BP-221994088-172.31.14.131-1692327744304] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 03:02:26,742 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/cluster_e68948b9-c4d2-07ea-5009-26cb263e42b5/dfs/data/data4/current/BP-221994088-172.31.14.131-1692327744304] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 03:02:26,743 WARN [Listener at localhost/45731] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-18 03:02:26,746 INFO [Listener at localhost/45731] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 03:02:26,849 WARN [BP-221994088-172.31.14.131-1692327744304 heartbeating to localhost/127.0.0.1:46109] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-18 03:02:26,849 WARN [BP-221994088-172.31.14.131-1692327744304 heartbeating to localhost/127.0.0.1:46109] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-221994088-172.31.14.131-1692327744304 (Datanode Uuid af1d0b66-1666-4ab0-862c-b086f5042729) service to localhost/127.0.0.1:46109 2023-08-18 03:02:26,850 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/cluster_e68948b9-c4d2-07ea-5009-26cb263e42b5/dfs/data/data1/current/BP-221994088-172.31.14.131-1692327744304] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 03:02:26,850 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/66835423-3681-5126-2e2c-22f53fad1e6e/cluster_e68948b9-c4d2-07ea-5009-26cb263e42b5/dfs/data/data2/current/BP-221994088-172.31.14.131-1692327744304] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-18 03:02:26,861 INFO [Listener at localhost/45731] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-18 03:02:26,972 INFO [Listener at localhost/45731] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-18 03:02:26,983 INFO [Listener at localhost/45731] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-18 03:02:26,991 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-18 03:02:26,993 INFO [Listener at localhost/45731] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=101 (was 76) - Thread LEAK? -, OpenFileDescriptor=485 (was 459) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=104 (was 104), ProcessCount=172 (was 172), AvailableMemoryMB=1740 (was 1881)