2023-09-04 05:59:55,373 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1 2023-09-04 05:59:55,387 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.regionserver.wal.TestAsyncLogRolling timeout: 13 mins 2023-09-04 05:59:55,418 INFO [Time-limited test] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=165, ProcessCount=166, AvailableMemoryMB=9236 2023-09-04 05:59:55,425 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-09-04 05:59:55,425 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a, deleteOnExit=true 2023-09-04 05:59:55,426 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-09-04 05:59:55,426 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/test.cache.data in system properties and HBase conf 2023-09-04 05:59:55,427 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/hadoop.tmp.dir in system properties and HBase conf 2023-09-04 05:59:55,427 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/hadoop.log.dir in system properties and HBase conf 2023-09-04 05:59:55,427 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/mapreduce.cluster.local.dir in system properties and HBase conf 2023-09-04 05:59:55,428 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-09-04 05:59:55,428 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-09-04 05:59:55,546 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-09-04 05:59:55,920 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-09-04 05:59:55,924 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-09-04 05:59:55,924 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-09-04 05:59:55,924 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-09-04 05:59:55,924 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 05:59:55,925 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-09-04 05:59:55,925 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-09-04 05:59:55,925 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 05:59:55,926 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 05:59:55,926 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-09-04 05:59:55,926 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/nfs.dump.dir in system properties and HBase conf 2023-09-04 05:59:55,926 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/java.io.tmpdir in system properties and HBase conf 2023-09-04 05:59:55,927 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 05:59:55,927 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-09-04 05:59:55,927 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-09-04 05:59:56,396 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 05:59:56,400 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 05:59:56,683 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-09-04 05:59:56,841 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-09-04 05:59:56,856 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 05:59:56,892 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 05:59:56,924 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/java.io.tmpdir/Jetty_localhost_39803_hdfs____pbk81r/webapp 2023-09-04 05:59:57,059 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39803 2023-09-04 05:59:57,077 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 05:59:57,078 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 05:59:57,563 WARN [Listener at localhost/42011] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 05:59:57,627 WARN [Listener at localhost/42011] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 05:59:57,646 WARN [Listener at localhost/42011] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 05:59:57,653 INFO [Listener at localhost/42011] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 05:59:57,659 INFO [Listener at localhost/42011] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/java.io.tmpdir/Jetty_localhost_34583_datanode____785ee/webapp 2023-09-04 05:59:57,754 INFO [Listener at localhost/42011] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34583 2023-09-04 05:59:58,053 WARN [Listener at localhost/36399] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 05:59:58,063 WARN [Listener at localhost/36399] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 05:59:58,066 WARN [Listener at localhost/36399] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 05:59:58,068 INFO [Listener at localhost/36399] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 05:59:58,072 INFO [Listener at localhost/36399] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/java.io.tmpdir/Jetty_localhost_33569_datanode____.3qxg3l/webapp 2023-09-04 05:59:58,167 INFO [Listener at localhost/36399] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33569 2023-09-04 05:59:58,176 WARN [Listener at localhost/44875] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 05:59:58,490 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x99cd50b629ebdb19: Processing first storage report for DS-83346a05-6c62-4509-bf14-60e3746d1304 from datanode b6023de8-7e06-4f53-a488-ff54e50d0cdd 2023-09-04 05:59:58,491 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x99cd50b629ebdb19: from storage DS-83346a05-6c62-4509-bf14-60e3746d1304 node DatanodeRegistration(127.0.0.1:40895, datanodeUuid=b6023de8-7e06-4f53-a488-ff54e50d0cdd, infoPort=33169, infoSecurePort=0, ipcPort=36399, storageInfo=lv=-57;cid=testClusterID;nsid=321774033;c=1693807196473), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-09-04 05:59:58,491 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4bbbda39dfca3472: Processing first storage report for DS-e4341291-06c9-49f0-b7b0-cb6586ad625f from datanode 32d93fbe-e3a0-4b19-a3c3-6c2b751b4e0f 2023-09-04 05:59:58,491 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4bbbda39dfca3472: from storage DS-e4341291-06c9-49f0-b7b0-cb6586ad625f node DatanodeRegistration(127.0.0.1:38753, datanodeUuid=32d93fbe-e3a0-4b19-a3c3-6c2b751b4e0f, infoPort=40717, infoSecurePort=0, ipcPort=44875, storageInfo=lv=-57;cid=testClusterID;nsid=321774033;c=1693807196473), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 05:59:58,491 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x99cd50b629ebdb19: Processing first storage report for DS-f1ba5d20-a008-4248-a37c-2eb10f4aacf5 from datanode b6023de8-7e06-4f53-a488-ff54e50d0cdd 2023-09-04 05:59:58,492 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x99cd50b629ebdb19: from storage DS-f1ba5d20-a008-4248-a37c-2eb10f4aacf5 node DatanodeRegistration(127.0.0.1:40895, datanodeUuid=b6023de8-7e06-4f53-a488-ff54e50d0cdd, infoPort=33169, infoSecurePort=0, ipcPort=36399, storageInfo=lv=-57;cid=testClusterID;nsid=321774033;c=1693807196473), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 05:59:58,492 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4bbbda39dfca3472: Processing first storage report for DS-582b401c-1923-4de2-af87-0a1ca7da5529 from datanode 32d93fbe-e3a0-4b19-a3c3-6c2b751b4e0f 2023-09-04 05:59:58,492 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4bbbda39dfca3472: from storage DS-582b401c-1923-4de2-af87-0a1ca7da5529 node DatanodeRegistration(127.0.0.1:38753, datanodeUuid=32d93fbe-e3a0-4b19-a3c3-6c2b751b4e0f, infoPort=40717, infoSecurePort=0, ipcPort=44875, storageInfo=lv=-57;cid=testClusterID;nsid=321774033;c=1693807196473), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 05:59:58,549 DEBUG [Listener at localhost/44875] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1 2023-09-04 05:59:58,611 INFO [Listener at localhost/44875] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a/zookeeper_0, clientPort=55230, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-09-04 05:59:58,624 INFO [Listener at localhost/44875] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=55230 2023-09-04 05:59:58,632 INFO [Listener at localhost/44875] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 05:59:58,634 INFO [Listener at localhost/44875] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 05:59:59,313 INFO [Listener at localhost/44875] util.FSUtils(471): Created version file at hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914 with version=8 2023-09-04 05:59:59,314 INFO [Listener at localhost/44875] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/hbase-staging 2023-09-04 05:59:59,642 INFO [Listener at localhost/44875] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-09-04 06:00:00,136 INFO [Listener at localhost/44875] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 06:00:00,168 INFO [Listener at localhost/44875] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:00:00,168 INFO [Listener at localhost/44875] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 06:00:00,169 INFO [Listener at localhost/44875] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 06:00:00,169 INFO [Listener at localhost/44875] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:00:00,169 INFO [Listener at localhost/44875] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 06:00:00,309 INFO [Listener at localhost/44875] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 06:00:00,382 DEBUG [Listener at localhost/44875] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-09-04 06:00:00,478 INFO [Listener at localhost/44875] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37663 2023-09-04 06:00:00,488 INFO [Listener at localhost/44875] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:00:00,490 INFO [Listener at localhost/44875] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:00:00,514 INFO [Listener at localhost/44875] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37663 connecting to ZooKeeper ensemble=127.0.0.1:55230 2023-09-04 06:00:00,553 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:376630x0, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 06:00:00,555 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:37663-0x1007585f92b0000 connected 2023-09-04 06:00:00,585 DEBUG [Listener at localhost/44875] zookeeper.ZKUtil(164): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 06:00:00,586 DEBUG [Listener at localhost/44875] zookeeper.ZKUtil(164): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:00:00,589 DEBUG [Listener at localhost/44875] zookeeper.ZKUtil(164): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 06:00:00,597 DEBUG [Listener at localhost/44875] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37663 2023-09-04 06:00:00,597 DEBUG [Listener at localhost/44875] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37663 2023-09-04 06:00:00,598 DEBUG [Listener at localhost/44875] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37663 2023-09-04 06:00:00,598 DEBUG [Listener at localhost/44875] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37663 2023-09-04 06:00:00,599 DEBUG [Listener at localhost/44875] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37663 2023-09-04 06:00:00,605 INFO [Listener at localhost/44875] master.HMaster(444): hbase.rootdir=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914, hbase.cluster.distributed=false 2023-09-04 06:00:00,676 INFO [Listener at localhost/44875] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 06:00:00,677 INFO [Listener at localhost/44875] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:00:00,677 INFO [Listener at localhost/44875] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 06:00:00,677 INFO [Listener at localhost/44875] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 06:00:00,677 INFO [Listener at localhost/44875] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:00:00,677 INFO [Listener at localhost/44875] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 06:00:00,682 INFO [Listener at localhost/44875] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 06:00:00,686 INFO [Listener at localhost/44875] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41993 2023-09-04 06:00:00,688 INFO [Listener at localhost/44875] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-04 06:00:00,694 DEBUG [Listener at localhost/44875] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-09-04 06:00:00,696 INFO [Listener at localhost/44875] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:00:00,698 INFO [Listener at localhost/44875] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:00:00,700 INFO [Listener at localhost/44875] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41993 connecting to ZooKeeper ensemble=127.0.0.1:55230 2023-09-04 06:00:00,704 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): regionserver:419930x0, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 06:00:00,704 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41993-0x1007585f92b0001 connected 2023-09-04 06:00:00,705 DEBUG [Listener at localhost/44875] zookeeper.ZKUtil(164): regionserver:41993-0x1007585f92b0001, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 06:00:00,706 DEBUG [Listener at localhost/44875] zookeeper.ZKUtil(164): regionserver:41993-0x1007585f92b0001, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:00:00,707 DEBUG [Listener at localhost/44875] zookeeper.ZKUtil(164): regionserver:41993-0x1007585f92b0001, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 06:00:00,707 DEBUG [Listener at localhost/44875] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41993 2023-09-04 06:00:00,709 DEBUG [Listener at localhost/44875] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41993 2023-09-04 06:00:00,709 DEBUG [Listener at localhost/44875] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41993 2023-09-04 06:00:00,710 DEBUG [Listener at localhost/44875] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41993 2023-09-04 06:00:00,710 DEBUG [Listener at localhost/44875] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41993 2023-09-04 06:00:00,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,37663,1693807199468 2023-09-04 06:00:00,722 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 06:00:00,724 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,37663,1693807199468 2023-09-04 06:00:00,743 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 06:00:00,743 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): regionserver:41993-0x1007585f92b0001, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 06:00:00,743 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:00,744 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 06:00:00,745 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,37663,1693807199468 from backup master directory 2023-09-04 06:00:00,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 06:00:00,748 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,37663,1693807199468 2023-09-04 06:00:00,748 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 06:00:00,749 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 06:00:00,749 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,37663,1693807199468 2023-09-04 06:00:00,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-09-04 06:00:00,753 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-09-04 06:00:00,841 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/hbase.id with ID: 968af024-678d-4ede-8f31-1e48b3c7682b 2023-09-04 06:00:00,885 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:00:00,900 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:00,946 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x580f8e8b to 127.0.0.1:55230 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 06:00:00,980 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3a187566, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 06:00:01,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-04 06:00:01,005 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-09-04 06:00:01,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 06:00:01,050 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/data/master/store-tmp 2023-09-04 06:00:01,085 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:01,085 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 06:00:01,085 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:00:01,086 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:00:01,086 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 06:00:01,086 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:00:01,086 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:00:01,086 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 06:00:01,088 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/WALs/jenkins-hbase4.apache.org,37663,1693807199468 2023-09-04 06:00:01,109 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37663%2C1693807199468, suffix=, logDir=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/WALs/jenkins-hbase4.apache.org,37663,1693807199468, archiveDir=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/oldWALs, maxLogs=10 2023-09-04 06:00:01,141 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-09-04 06:00:01,141 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-09-04 06:00:01,143 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:190) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:719) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:01,174 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:417) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:01,176 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40895,DS-83346a05-6c62-4509-bf14-60e3746d1304,DISK] 2023-09-04 06:00:01,176 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38753,DS-e4341291-06c9-49f0-b7b0-cb6586ad625f,DISK] 2023-09-04 06:00:01,183 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:01,252 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/WALs/jenkins-hbase4.apache.org,37663,1693807199468/jenkins-hbase4.apache.org%2C37663%2C1693807199468.1693807201120 2023-09-04 06:00:01,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38753,DS-e4341291-06c9-49f0-b7b0-cb6586ad625f,DISK], DatanodeInfoWithStorage[127.0.0.1:40895,DS-83346a05-6c62-4509-bf14-60e3746d1304,DISK]] 2023-09-04 06:00:01,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-09-04 06:00:01,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:01,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:00:01,259 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:00:01,329 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:00:01,338 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-09-04 06:00:01,369 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-09-04 06:00:01,381 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:01,387 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:00:01,389 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:00:01,404 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:00:01,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 06:00:01,410 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=810742, jitterRate=0.030912742018699646}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 06:00:01,410 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 06:00:01,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-09-04 06:00:01,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-09-04 06:00:01,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-09-04 06:00:01,432 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-09-04 06:00:01,434 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 1 msec 2023-09-04 06:00:01,466 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 32 msec 2023-09-04 06:00:01,466 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-09-04 06:00:01,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-09-04 06:00:01,497 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-09-04 06:00:01,522 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-09-04 06:00:01,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-09-04 06:00:01,528 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-09-04 06:00:01,533 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-09-04 06:00:01,537 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-09-04 06:00:01,540 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:01,542 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-09-04 06:00:01,542 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-09-04 06:00:01,554 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-09-04 06:00:01,558 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 06:00:01,558 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): regionserver:41993-0x1007585f92b0001, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 06:00:01,558 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:01,559 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,37663,1693807199468, sessionid=0x1007585f92b0000, setting cluster-up flag (Was=false) 2023-09-04 06:00:01,575 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:01,580 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-09-04 06:00:01,581 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37663,1693807199468 2023-09-04 06:00:01,587 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:01,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-09-04 06:00:01,594 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37663,1693807199468 2023-09-04 06:00:01,597 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/.hbase-snapshot/.tmp 2023-09-04 06:00:01,614 INFO [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(951): ClusterId : 968af024-678d-4ede-8f31-1e48b3c7682b 2023-09-04 06:00:01,619 DEBUG [RS:0;jenkins-hbase4:41993] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-09-04 06:00:01,624 DEBUG [RS:0;jenkins-hbase4:41993] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-09-04 06:00:01,624 DEBUG [RS:0;jenkins-hbase4:41993] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-09-04 06:00:01,627 DEBUG [RS:0;jenkins-hbase4:41993] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-09-04 06:00:01,627 DEBUG [RS:0;jenkins-hbase4:41993] zookeeper.ReadOnlyZKClient(139): Connect 0x652a9a0c to 127.0.0.1:55230 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 06:00:01,632 DEBUG [RS:0;jenkins-hbase4:41993] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6bed78d9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 06:00:01,633 DEBUG [RS:0;jenkins-hbase4:41993] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6fd2f17c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 06:00:01,656 DEBUG [RS:0;jenkins-hbase4:41993] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:41993 2023-09-04 06:00:01,660 INFO [RS:0;jenkins-hbase4:41993] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-09-04 06:00:01,660 INFO [RS:0;jenkins-hbase4:41993] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-09-04 06:00:01,660 DEBUG [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(1022): About to register with Master. 2023-09-04 06:00:01,663 INFO [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,37663,1693807199468 with isa=jenkins-hbase4.apache.org/172.31.14.131:41993, startcode=1693807200675 2023-09-04 06:00:01,682 DEBUG [RS:0;jenkins-hbase4:41993] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-09-04 06:00:01,723 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-09-04 06:00:01,733 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60657, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-09-04 06:00:01,734 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:00:01,734 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:00:01,734 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:00:01,734 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:00:01,735 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-09-04 06:00:01,735 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:01,735 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 06:00:01,735 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:01,736 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693807231736 2023-09-04 06:00:01,739 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-09-04 06:00:01,742 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37663] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 06:00:01,743 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 06:00:01,744 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-09-04 06:00:01,749 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 06:00:01,752 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-09-04 06:00:01,758 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-09-04 06:00:01,759 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-09-04 06:00:01,759 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-09-04 06:00:01,759 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-09-04 06:00:01,760 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:01,762 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-09-04 06:00:01,763 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-09-04 06:00:01,763 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-09-04 06:00:01,766 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-09-04 06:00:01,767 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-09-04 06:00:01,768 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693807201768,5,FailOnTimeoutGroup] 2023-09-04 06:00:01,768 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693807201768,5,FailOnTimeoutGroup] 2023-09-04 06:00:01,769 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:01,769 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-09-04 06:00:01,771 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:01,771 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:01,782 DEBUG [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(2830): Master is not running yet 2023-09-04 06:00:01,782 WARN [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-09-04 06:00:01,784 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 06:00:01,785 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 06:00:01,786 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914 2023-09-04 06:00:01,810 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:01,812 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 06:00:01,815 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/info 2023-09-04 06:00:01,816 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 06:00:01,817 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:01,817 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 06:00:01,820 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/rep_barrier 2023-09-04 06:00:01,820 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 06:00:01,821 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:01,822 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 06:00:01,824 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/table 2023-09-04 06:00:01,824 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 06:00:01,825 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:01,827 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740 2023-09-04 06:00:01,828 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740 2023-09-04 06:00:01,831 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 06:00:01,834 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 06:00:01,837 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 06:00:01,838 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=722802, jitterRate=-0.08091004192829132}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 06:00:01,838 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 06:00:01,838 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 06:00:01,838 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 06:00:01,839 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 06:00:01,839 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 06:00:01,839 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 06:00:01,839 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 06:00:01,840 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 06:00:01,845 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 06:00:01,845 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-09-04 06:00:01,853 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-09-04 06:00:01,865 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-09-04 06:00:01,867 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-09-04 06:00:01,884 INFO [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,37663,1693807199468 with isa=jenkins-hbase4.apache.org/172.31.14.131:41993, startcode=1693807200675 2023-09-04 06:00:01,889 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37663] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41993,1693807200675 2023-09-04 06:00:01,895 DEBUG [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914 2023-09-04 06:00:01,895 DEBUG [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:42011 2023-09-04 06:00:01,895 DEBUG [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-09-04 06:00:01,904 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 06:00:01,905 DEBUG [RS:0;jenkins-hbase4:41993] zookeeper.ZKUtil(162): regionserver:41993-0x1007585f92b0001, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41993,1693807200675 2023-09-04 06:00:01,906 WARN [RS:0;jenkins-hbase4:41993] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 06:00:01,906 INFO [RS:0;jenkins-hbase4:41993] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 06:00:01,906 DEBUG [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(1948): logDir=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675 2023-09-04 06:00:01,907 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41993,1693807200675] 2023-09-04 06:00:01,915 DEBUG [RS:0;jenkins-hbase4:41993] zookeeper.ZKUtil(162): regionserver:41993-0x1007585f92b0001, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41993,1693807200675 2023-09-04 06:00:01,928 DEBUG [RS:0;jenkins-hbase4:41993] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-09-04 06:00:01,940 INFO [RS:0;jenkins-hbase4:41993] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-09-04 06:00:01,960 INFO [RS:0;jenkins-hbase4:41993] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-09-04 06:00:01,963 INFO [RS:0;jenkins-hbase4:41993] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-09-04 06:00:01,963 INFO [RS:0;jenkins-hbase4:41993] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:01,964 INFO [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-09-04 06:00:01,972 INFO [RS:0;jenkins-hbase4:41993] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:01,972 DEBUG [RS:0;jenkins-hbase4:41993] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:01,972 DEBUG [RS:0;jenkins-hbase4:41993] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:01,973 DEBUG [RS:0;jenkins-hbase4:41993] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:01,973 DEBUG [RS:0;jenkins-hbase4:41993] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:01,973 DEBUG [RS:0;jenkins-hbase4:41993] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:01,973 DEBUG [RS:0;jenkins-hbase4:41993] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 06:00:01,973 DEBUG [RS:0;jenkins-hbase4:41993] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:01,973 DEBUG [RS:0;jenkins-hbase4:41993] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:01,974 DEBUG [RS:0;jenkins-hbase4:41993] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:01,974 DEBUG [RS:0;jenkins-hbase4:41993] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:01,975 INFO [RS:0;jenkins-hbase4:41993] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:01,975 INFO [RS:0;jenkins-hbase4:41993] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:01,975 INFO [RS:0;jenkins-hbase4:41993] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:01,992 INFO [RS:0;jenkins-hbase4:41993] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-09-04 06:00:01,994 INFO [RS:0;jenkins-hbase4:41993] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41993,1693807200675-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:02,010 INFO [RS:0;jenkins-hbase4:41993] regionserver.Replication(203): jenkins-hbase4.apache.org,41993,1693807200675 started 2023-09-04 06:00:02,010 INFO [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,41993,1693807200675, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41993, sessionid=0x1007585f92b0001 2023-09-04 06:00:02,010 DEBUG [RS:0;jenkins-hbase4:41993] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-09-04 06:00:02,010 DEBUG [RS:0;jenkins-hbase4:41993] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41993,1693807200675 2023-09-04 06:00:02,010 DEBUG [RS:0;jenkins-hbase4:41993] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41993,1693807200675' 2023-09-04 06:00:02,011 DEBUG [RS:0;jenkins-hbase4:41993] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 06:00:02,011 DEBUG [RS:0;jenkins-hbase4:41993] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 06:00:02,012 DEBUG [RS:0;jenkins-hbase4:41993] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-09-04 06:00:02,012 DEBUG [RS:0;jenkins-hbase4:41993] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-09-04 06:00:02,012 DEBUG [RS:0;jenkins-hbase4:41993] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41993,1693807200675 2023-09-04 06:00:02,012 DEBUG [RS:0;jenkins-hbase4:41993] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41993,1693807200675' 2023-09-04 06:00:02,012 DEBUG [RS:0;jenkins-hbase4:41993] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-09-04 06:00:02,013 DEBUG [RS:0;jenkins-hbase4:41993] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-09-04 06:00:02,013 DEBUG [RS:0;jenkins-hbase4:41993] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-09-04 06:00:02,013 INFO [RS:0;jenkins-hbase4:41993] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-09-04 06:00:02,014 INFO [RS:0;jenkins-hbase4:41993] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-09-04 06:00:02,019 DEBUG [jenkins-hbase4:37663] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-09-04 06:00:02,021 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41993,1693807200675, state=OPENING 2023-09-04 06:00:02,028 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-09-04 06:00:02,030 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:02,031 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 06:00:02,034 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41993,1693807200675}] 2023-09-04 06:00:02,124 INFO [RS:0;jenkins-hbase4:41993] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41993%2C1693807200675, suffix=, logDir=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675, archiveDir=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/oldWALs, maxLogs=32 2023-09-04 06:00:02,145 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38753,DS-e4341291-06c9-49f0-b7b0-cb6586ad625f,DISK] 2023-09-04 06:00:02,145 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40895,DS-83346a05-6c62-4509-bf14-60e3746d1304,DISK] 2023-09-04 06:00:02,149 INFO [RS:0;jenkins-hbase4:41993] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.1693807202127 2023-09-04 06:00:02,149 DEBUG [RS:0;jenkins-hbase4:41993] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38753,DS-e4341291-06c9-49f0-b7b0-cb6586ad625f,DISK], DatanodeInfoWithStorage[127.0.0.1:40895,DS-83346a05-6c62-4509-bf14-60e3746d1304,DISK]] 2023-09-04 06:00:02,220 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41993,1693807200675 2023-09-04 06:00:02,223 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-09-04 06:00:02,227 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37710, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-09-04 06:00:02,241 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-09-04 06:00:02,242 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 06:00:02,246 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41993%2C1693807200675.meta, suffix=.meta, logDir=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675, archiveDir=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/oldWALs, maxLogs=32 2023-09-04 06:00:02,265 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40895,DS-83346a05-6c62-4509-bf14-60e3746d1304,DISK] 2023-09-04 06:00:02,265 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38753,DS-e4341291-06c9-49f0-b7b0-cb6586ad625f,DISK] 2023-09-04 06:00:02,269 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.meta.1693807202248.meta 2023-09-04 06:00:02,272 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40895,DS-83346a05-6c62-4509-bf14-60e3746d1304,DISK], DatanodeInfoWithStorage[127.0.0.1:38753,DS-e4341291-06c9-49f0-b7b0-cb6586ad625f,DISK]] 2023-09-04 06:00:02,273 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-09-04 06:00:02,275 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-09-04 06:00:02,293 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-09-04 06:00:02,299 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-09-04 06:00:02,305 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-09-04 06:00:02,305 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:02,305 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-09-04 06:00:02,306 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-09-04 06:00:02,309 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 06:00:02,311 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/info 2023-09-04 06:00:02,311 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/info 2023-09-04 06:00:02,312 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 06:00:02,313 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:02,313 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 06:00:02,314 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/rep_barrier 2023-09-04 06:00:02,315 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/rep_barrier 2023-09-04 06:00:02,315 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 06:00:02,316 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:02,316 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 06:00:02,318 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/table 2023-09-04 06:00:02,318 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/table 2023-09-04 06:00:02,319 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 06:00:02,319 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:02,321 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740 2023-09-04 06:00:02,324 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740 2023-09-04 06:00:02,328 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 06:00:02,330 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 06:00:02,332 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=875362, jitterRate=0.1130814403295517}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 06:00:02,332 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 06:00:02,342 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693807202212 2023-09-04 06:00:02,361 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-09-04 06:00:02,362 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-09-04 06:00:02,363 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41993,1693807200675, state=OPEN 2023-09-04 06:00:02,366 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-09-04 06:00:02,366 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 06:00:02,372 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-09-04 06:00:02,372 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41993,1693807200675 in 332 msec 2023-09-04 06:00:02,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-09-04 06:00:02,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 520 msec 2023-09-04 06:00:02,384 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 738 msec 2023-09-04 06:00:02,385 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693807202385, completionTime=-1 2023-09-04 06:00:02,385 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-09-04 06:00:02,385 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-09-04 06:00:02,455 DEBUG [hconnection-0x42990faa-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 06:00:02,458 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37722, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 06:00:02,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-09-04 06:00:02,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693807262474 2023-09-04 06:00:02,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693807322474 2023-09-04 06:00:02,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 88 msec 2023-09-04 06:00:02,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37663,1693807199468-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:02,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37663,1693807199468-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:02,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37663,1693807199468-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:02,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:37663, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:02,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:02,507 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-09-04 06:00:02,515 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-09-04 06:00:02,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 06:00:02,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-09-04 06:00:02,530 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-09-04 06:00:02,533 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-04 06:00:02,560 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/.tmp/data/hbase/namespace/4486abc3f5c29833f5dc8026cdf7364e 2023-09-04 06:00:02,562 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/.tmp/data/hbase/namespace/4486abc3f5c29833f5dc8026cdf7364e empty. 2023-09-04 06:00:02,563 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/.tmp/data/hbase/namespace/4486abc3f5c29833f5dc8026cdf7364e 2023-09-04 06:00:02,563 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-09-04 06:00:02,591 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-09-04 06:00:02,593 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 4486abc3f5c29833f5dc8026cdf7364e, NAME => 'hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/.tmp 2023-09-04 06:00:02,612 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:02,612 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 4486abc3f5c29833f5dc8026cdf7364e, disabling compactions & flushes 2023-09-04 06:00:02,612 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e. 2023-09-04 06:00:02,612 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e. 2023-09-04 06:00:02,612 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e. after waiting 0 ms 2023-09-04 06:00:02,612 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e. 2023-09-04 06:00:02,612 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e. 2023-09-04 06:00:02,612 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 4486abc3f5c29833f5dc8026cdf7364e: 2023-09-04 06:00:02,617 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-09-04 06:00:02,638 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693807202620"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693807202620"}]},"ts":"1693807202620"} 2023-09-04 06:00:02,672 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-04 06:00:02,674 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-04 06:00:02,681 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693807202674"}]},"ts":"1693807202674"} 2023-09-04 06:00:02,685 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-09-04 06:00:02,695 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=4486abc3f5c29833f5dc8026cdf7364e, ASSIGN}] 2023-09-04 06:00:02,699 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=4486abc3f5c29833f5dc8026cdf7364e, ASSIGN 2023-09-04 06:00:02,701 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=4486abc3f5c29833f5dc8026cdf7364e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41993,1693807200675; forceNewPlan=false, retain=false 2023-09-04 06:00:02,852 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=4486abc3f5c29833f5dc8026cdf7364e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41993,1693807200675 2023-09-04 06:00:02,852 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693807202852"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693807202852"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693807202852"}]},"ts":"1693807202852"} 2023-09-04 06:00:02,857 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 4486abc3f5c29833f5dc8026cdf7364e, server=jenkins-hbase4.apache.org,41993,1693807200675}] 2023-09-04 06:00:03,020 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e. 2023-09-04 06:00:03,021 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4486abc3f5c29833f5dc8026cdf7364e, NAME => 'hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e.', STARTKEY => '', ENDKEY => ''} 2023-09-04 06:00:03,022 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 4486abc3f5c29833f5dc8026cdf7364e 2023-09-04 06:00:03,022 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:03,022 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4486abc3f5c29833f5dc8026cdf7364e 2023-09-04 06:00:03,022 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4486abc3f5c29833f5dc8026cdf7364e 2023-09-04 06:00:03,024 INFO [StoreOpener-4486abc3f5c29833f5dc8026cdf7364e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 4486abc3f5c29833f5dc8026cdf7364e 2023-09-04 06:00:03,026 DEBUG [StoreOpener-4486abc3f5c29833f5dc8026cdf7364e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/namespace/4486abc3f5c29833f5dc8026cdf7364e/info 2023-09-04 06:00:03,026 DEBUG [StoreOpener-4486abc3f5c29833f5dc8026cdf7364e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/namespace/4486abc3f5c29833f5dc8026cdf7364e/info 2023-09-04 06:00:03,027 INFO [StoreOpener-4486abc3f5c29833f5dc8026cdf7364e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4486abc3f5c29833f5dc8026cdf7364e columnFamilyName info 2023-09-04 06:00:03,028 INFO [StoreOpener-4486abc3f5c29833f5dc8026cdf7364e-1] regionserver.HStore(310): Store=4486abc3f5c29833f5dc8026cdf7364e/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:03,029 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/namespace/4486abc3f5c29833f5dc8026cdf7364e 2023-09-04 06:00:03,030 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/namespace/4486abc3f5c29833f5dc8026cdf7364e 2023-09-04 06:00:03,034 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4486abc3f5c29833f5dc8026cdf7364e 2023-09-04 06:00:03,038 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/namespace/4486abc3f5c29833f5dc8026cdf7364e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 06:00:03,039 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4486abc3f5c29833f5dc8026cdf7364e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=712446, jitterRate=-0.09407849609851837}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 06:00:03,039 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4486abc3f5c29833f5dc8026cdf7364e: 2023-09-04 06:00:03,041 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e., pid=6, masterSystemTime=1693807203011 2023-09-04 06:00:03,045 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e. 2023-09-04 06:00:03,045 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e. 2023-09-04 06:00:03,047 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=4486abc3f5c29833f5dc8026cdf7364e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41993,1693807200675 2023-09-04 06:00:03,047 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693807203046"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693807203046"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693807203046"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693807203046"}]},"ts":"1693807203046"} 2023-09-04 06:00:03,055 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-09-04 06:00:03,055 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 4486abc3f5c29833f5dc8026cdf7364e, server=jenkins-hbase4.apache.org,41993,1693807200675 in 194 msec 2023-09-04 06:00:03,059 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-09-04 06:00:03,059 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=4486abc3f5c29833f5dc8026cdf7364e, ASSIGN in 360 msec 2023-09-04 06:00:03,060 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-04 06:00:03,061 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693807203061"}]},"ts":"1693807203061"} 2023-09-04 06:00:03,065 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-09-04 06:00:03,069 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-09-04 06:00:03,072 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 551 msec 2023-09-04 06:00:03,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-09-04 06:00:03,131 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-09-04 06:00:03,131 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:03,171 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-09-04 06:00:03,188 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 06:00:03,195 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 33 msec 2023-09-04 06:00:03,209 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-09-04 06:00:03,224 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 06:00:03,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 20 msec 2023-09-04 06:00:03,236 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-09-04 06:00:03,239 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-09-04 06:00:03,239 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.490sec 2023-09-04 06:00:03,241 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-09-04 06:00:03,243 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-09-04 06:00:03,243 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-09-04 06:00:03,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37663,1693807199468-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-09-04 06:00:03,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37663,1693807199468-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-09-04 06:00:03,255 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-09-04 06:00:03,320 DEBUG [Listener at localhost/44875] zookeeper.ReadOnlyZKClient(139): Connect 0x0f1dfe88 to 127.0.0.1:55230 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 06:00:03,326 DEBUG [Listener at localhost/44875] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@c21dbf9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 06:00:03,379 DEBUG [hconnection-0x6b78fdb9-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 06:00:03,391 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37726, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 06:00:03,402 INFO [Listener at localhost/44875] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,37663,1693807199468 2023-09-04 06:00:03,402 INFO [Listener at localhost/44875] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:00:03,413 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-09-04 06:00:03,413 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:03,414 INFO [Listener at localhost/44875] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-09-04 06:00:03,428 WARN [Listener at localhost/44875] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 06:00:03,431 WARN [Listener at localhost/44875] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 06:00:03,433 INFO [Listener at localhost/44875] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 06:00:03,437 INFO [Listener at localhost/44875] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/java.io.tmpdir/Jetty_localhost_46177_datanode____.y1toju/webapp 2023-09-04 06:00:03,535 INFO [Listener at localhost/44875] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46177 2023-09-04 06:00:03,546 WARN [Listener at localhost/41315] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 06:00:03,586 WARN [Listener at localhost/41315] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 06:00:03,589 WARN [Listener at localhost/41315] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 06:00:03,591 INFO [Listener at localhost/41315] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 06:00:03,595 INFO [Listener at localhost/41315] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/java.io.tmpdir/Jetty_localhost_34463_datanode____.80c5yh/webapp 2023-09-04 06:00:03,695 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x733c71bab33e8e6: Processing first storage report for DS-770909ef-a6eb-46a2-a45d-541691e80c67 from datanode 01f55e7b-32c7-4705-b9e7-70560f25f10e 2023-09-04 06:00:03,696 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x733c71bab33e8e6: from storage DS-770909ef-a6eb-46a2-a45d-541691e80c67 node DatanodeRegistration(127.0.0.1:35711, datanodeUuid=01f55e7b-32c7-4705-b9e7-70560f25f10e, infoPort=37353, infoSecurePort=0, ipcPort=41315, storageInfo=lv=-57;cid=testClusterID;nsid=321774033;c=1693807196473), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:00:03,696 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x733c71bab33e8e6: Processing first storage report for DS-cd15723c-e2d4-41d5-bd8b-81545488a95f from datanode 01f55e7b-32c7-4705-b9e7-70560f25f10e 2023-09-04 06:00:03,696 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x733c71bab33e8e6: from storage DS-cd15723c-e2d4-41d5-bd8b-81545488a95f node DatanodeRegistration(127.0.0.1:35711, datanodeUuid=01f55e7b-32c7-4705-b9e7-70560f25f10e, infoPort=37353, infoSecurePort=0, ipcPort=41315, storageInfo=lv=-57;cid=testClusterID;nsid=321774033;c=1693807196473), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:00:03,706 INFO [Listener at localhost/41315] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34463 2023-09-04 06:00:03,714 WARN [Listener at localhost/33473] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 06:00:03,742 WARN [Listener at localhost/33473] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 06:00:03,748 WARN [Listener at localhost/33473] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 06:00:03,750 INFO [Listener at localhost/33473] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 06:00:03,756 INFO [Listener at localhost/33473] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/java.io.tmpdir/Jetty_localhost_40583_datanode____.eobpd1/webapp 2023-09-04 06:00:03,847 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd21899330e16906b: Processing first storage report for DS-e6b2c7c2-53e1-4219-8045-0839a0a827ba from datanode 09aefbf6-d44c-4fb8-b8b7-a521c07d3e76 2023-09-04 06:00:03,847 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd21899330e16906b: from storage DS-e6b2c7c2-53e1-4219-8045-0839a0a827ba node DatanodeRegistration(127.0.0.1:44339, datanodeUuid=09aefbf6-d44c-4fb8-b8b7-a521c07d3e76, infoPort=42775, infoSecurePort=0, ipcPort=33473, storageInfo=lv=-57;cid=testClusterID;nsid=321774033;c=1693807196473), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-09-04 06:00:03,847 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd21899330e16906b: Processing first storage report for DS-a3d48119-a79f-41eb-a5e2-7abf5e964c4d from datanode 09aefbf6-d44c-4fb8-b8b7-a521c07d3e76 2023-09-04 06:00:03,847 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd21899330e16906b: from storage DS-a3d48119-a79f-41eb-a5e2-7abf5e964c4d node DatanodeRegistration(127.0.0.1:44339, datanodeUuid=09aefbf6-d44c-4fb8-b8b7-a521c07d3e76, infoPort=42775, infoSecurePort=0, ipcPort=33473, storageInfo=lv=-57;cid=testClusterID;nsid=321774033;c=1693807196473), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:00:03,872 INFO [Listener at localhost/33473] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40583 2023-09-04 06:00:03,884 WARN [Listener at localhost/46723] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 06:00:04,002 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd3aed637e580161b: Processing first storage report for DS-d3eadf9a-06ef-4066-97c7-62deb958d9bc from datanode d73cee86-9872-478e-b72f-008a0fbced21 2023-09-04 06:00:04,002 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd3aed637e580161b: from storage DS-d3eadf9a-06ef-4066-97c7-62deb958d9bc node DatanodeRegistration(127.0.0.1:41647, datanodeUuid=d73cee86-9872-478e-b72f-008a0fbced21, infoPort=44015, infoSecurePort=0, ipcPort=46723, storageInfo=lv=-57;cid=testClusterID;nsid=321774033;c=1693807196473), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-09-04 06:00:04,002 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd3aed637e580161b: Processing first storage report for DS-41367f5a-69bb-4ae2-b8a7-7370a408dda9 from datanode d73cee86-9872-478e-b72f-008a0fbced21 2023-09-04 06:00:04,002 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd3aed637e580161b: from storage DS-41367f5a-69bb-4ae2-b8a7-7370a408dda9 node DatanodeRegistration(127.0.0.1:41647, datanodeUuid=d73cee86-9872-478e-b72f-008a0fbced21, infoPort=44015, infoSecurePort=0, ipcPort=46723, storageInfo=lv=-57;cid=testClusterID;nsid=321774033;c=1693807196473), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:00:04,109 DEBUG [Listener at localhost/46723] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-09-04 06:00:04,112 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56590, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-09-04 06:00:04,121 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37663] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-09-04 06:00:04,122 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37663] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-09-04 06:00:04,125 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37663] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-04 06:00:04,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37663] procedure2.ProcedureExecutor(1028): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath 2023-09-04 06:00:04,130 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_PRE_OPERATION 2023-09-04 06:00:04,132 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-04 06:00:04,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37663] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRollOnDatanodeDeath" procId is: 9 2023-09-04 06:00:04,136 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/109b79e843fc9fd6517ada18445d914e 2023-09-04 06:00:04,138 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/109b79e843fc9fd6517ada18445d914e empty. 2023-09-04 06:00:04,140 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/109b79e843fc9fd6517ada18445d914e 2023-09-04 06:00:04,140 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRollOnDatanodeDeath regions 2023-09-04 06:00:04,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37663] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-04 06:00:04,162 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/.tabledesc/.tableinfo.0000000001 2023-09-04 06:00:04,164 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(7675): creating {ENCODED => 109b79e843fc9fd6517ada18445d914e, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/.tmp 2023-09-04 06:00:04,180 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:04,181 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1604): Closing 109b79e843fc9fd6517ada18445d914e, disabling compactions & flushes 2023-09-04 06:00:04,181 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e. 2023-09-04 06:00:04,181 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e. 2023-09-04 06:00:04,181 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e. after waiting 0 ms 2023-09-04 06:00:04,181 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e. 2023-09-04 06:00:04,181 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e. 2023-09-04 06:00:04,181 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1558): Region close journal for 109b79e843fc9fd6517ada18445d914e: 2023-09-04 06:00:04,185 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ADD_TO_META 2023-09-04 06:00:04,187 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1693807204187"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693807204187"}]},"ts":"1693807204187"} 2023-09-04 06:00:04,190 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-04 06:00:04,192 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-04 06:00:04,192 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693807204192"}]},"ts":"1693807204192"} 2023-09-04 06:00:04,195 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLING in hbase:meta 2023-09-04 06:00:04,199 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=109b79e843fc9fd6517ada18445d914e, ASSIGN}] 2023-09-04 06:00:04,201 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=109b79e843fc9fd6517ada18445d914e, ASSIGN 2023-09-04 06:00:04,202 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=109b79e843fc9fd6517ada18445d914e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41993,1693807200675; forceNewPlan=false, retain=false 2023-09-04 06:00:04,354 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=109b79e843fc9fd6517ada18445d914e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41993,1693807200675 2023-09-04 06:00:04,355 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1693807204354"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693807204354"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693807204354"}]},"ts":"1693807204354"} 2023-09-04 06:00:04,359 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 109b79e843fc9fd6517ada18445d914e, server=jenkins-hbase4.apache.org,41993,1693807200675}] 2023-09-04 06:00:04,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e. 2023-09-04 06:00:04,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 109b79e843fc9fd6517ada18445d914e, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e.', STARTKEY => '', ENDKEY => ''} 2023-09-04 06:00:04,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRollOnDatanodeDeath 109b79e843fc9fd6517ada18445d914e 2023-09-04 06:00:04,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:04,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 109b79e843fc9fd6517ada18445d914e 2023-09-04 06:00:04,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 109b79e843fc9fd6517ada18445d914e 2023-09-04 06:00:04,523 INFO [StoreOpener-109b79e843fc9fd6517ada18445d914e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 109b79e843fc9fd6517ada18445d914e 2023-09-04 06:00:04,525 DEBUG [StoreOpener-109b79e843fc9fd6517ada18445d914e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/default/TestLogRolling-testLogRollOnDatanodeDeath/109b79e843fc9fd6517ada18445d914e/info 2023-09-04 06:00:04,525 DEBUG [StoreOpener-109b79e843fc9fd6517ada18445d914e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/default/TestLogRolling-testLogRollOnDatanodeDeath/109b79e843fc9fd6517ada18445d914e/info 2023-09-04 06:00:04,526 INFO [StoreOpener-109b79e843fc9fd6517ada18445d914e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 109b79e843fc9fd6517ada18445d914e columnFamilyName info 2023-09-04 06:00:04,526 INFO [StoreOpener-109b79e843fc9fd6517ada18445d914e-1] regionserver.HStore(310): Store=109b79e843fc9fd6517ada18445d914e/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:04,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/default/TestLogRolling-testLogRollOnDatanodeDeath/109b79e843fc9fd6517ada18445d914e 2023-09-04 06:00:04,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/default/TestLogRolling-testLogRollOnDatanodeDeath/109b79e843fc9fd6517ada18445d914e 2023-09-04 06:00:04,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 109b79e843fc9fd6517ada18445d914e 2023-09-04 06:00:04,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/default/TestLogRolling-testLogRollOnDatanodeDeath/109b79e843fc9fd6517ada18445d914e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 06:00:04,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 109b79e843fc9fd6517ada18445d914e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=851237, jitterRate=0.0824039876461029}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 06:00:04,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 109b79e843fc9fd6517ada18445d914e: 2023-09-04 06:00:04,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e., pid=11, masterSystemTime=1693807204512 2023-09-04 06:00:04,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e. 2023-09-04 06:00:04,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e. 2023-09-04 06:00:04,542 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=109b79e843fc9fd6517ada18445d914e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41993,1693807200675 2023-09-04 06:00:04,542 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1693807204542"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693807204542"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693807204542"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693807204542"}]},"ts":"1693807204542"} 2023-09-04 06:00:04,548 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=11, resume processing ppid=10 2023-09-04 06:00:04,548 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 109b79e843fc9fd6517ada18445d914e, server=jenkins-hbase4.apache.org,41993,1693807200675 in 186 msec 2023-09-04 06:00:04,551 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=10, resume processing ppid=9 2023-09-04 06:00:04,552 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=109b79e843fc9fd6517ada18445d914e, ASSIGN in 349 msec 2023-09-04 06:00:04,553 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-04 06:00:04,553 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693807204553"}]},"ts":"1693807204553"} 2023-09-04 06:00:04,555 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLED in hbase:meta 2023-09-04 06:00:04,558 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_POST_OPERATION 2023-09-04 06:00:04,561 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath in 432 msec 2023-09-04 06:00:07,817 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-04 06:00:07,935 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-09-04 06:00:07,936 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-09-04 06:00:07,937 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRollOnDatanodeDeath' 2023-09-04 06:00:09,638 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-04 06:00:09,638 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-09-04 06:00:14,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37663] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-04 06:00:14,153 INFO [Listener at localhost/46723] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRollOnDatanodeDeath, procId: 9 completed 2023-09-04 06:00:14,153 DEBUG [Listener at localhost/46723] hbase.HBaseTestingUtility(3430): Waiting until all regions of table TestLogRolling-testLogRollOnDatanodeDeath get assigned. Timeout = 60000ms 2023-09-04 06:00:14,155 INFO [Listener at localhost/46723] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-09-04 06:00:14,159 INFO [Listener at localhost/46723] hbase.HBaseTestingUtility(3484): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned to meta. Checking AM states. 2023-09-04 06:00:14,160 INFO [Listener at localhost/46723] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-09-04 06:00:14,160 INFO [Listener at localhost/46723] hbase.HBaseTestingUtility(3504): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned. 2023-09-04 06:00:14,170 DEBUG [Listener at localhost/46723] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRollOnDatanodeDeath 2023-09-04 06:00:14,171 DEBUG [Listener at localhost/46723] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e. 2023-09-04 06:00:14,172 WARN [Listener at localhost/46723] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 06:00:14,175 INFO [Listener at localhost/46723] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:00:14,181 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1612968898_17 at /127.0.0.1:32908 [Receiving block BP-1630235210-172.31.14.131-1693807196473:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:40895:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:32908 dst: /127.0.0.1:40895 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:14,181 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_1110027845_17 at /127.0.0.1:32958 [Receiving block BP-1630235210-172.31.14.131-1693807196473:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:40895:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:32958 dst: /127.0.0.1:40895 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:14,181 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_1110027845_17 at /127.0.0.1:32946 [Receiving block BP-1630235210-172.31.14.131-1693807196473:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:40895:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:32946 dst: /127.0.0.1:40895 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:14,278 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1612968898_17 at /127.0.0.1:41870 [Receiving block BP-1630235210-172.31.14.131-1693807196473:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:38753:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:41870 dst: /127.0.0.1:38753 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:14,279 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_1110027845_17 at /127.0.0.1:41904 [Receiving block BP-1630235210-172.31.14.131-1693807196473:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:38753:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:41904 dst: /127.0.0.1:38753 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:14,279 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_1110027845_17 at /127.0.0.1:41892 [Receiving block BP-1630235210-172.31.14.131-1693807196473:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:38753:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:41892 dst: /127.0.0.1:38753 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:14,280 WARN [BP-1630235210-172.31.14.131-1693807196473 heartbeating to localhost/127.0.0.1:42011] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 06:00:14,281 WARN [BP-1630235210-172.31.14.131-1693807196473 heartbeating to localhost/127.0.0.1:42011] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1630235210-172.31.14.131-1693807196473 (Datanode Uuid 32d93fbe-e3a0-4b19-a3c3-6c2b751b4e0f) service to localhost/127.0.0.1:42011 2023-09-04 06:00:14,283 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a/dfs/data/data3/current/BP-1630235210-172.31.14.131-1693807196473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:00:14,283 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a/dfs/data/data4/current/BP-1630235210-172.31.14.131-1693807196473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:00:14,291 WARN [Listener at localhost/46723] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 06:00:14,293 WARN [Listener at localhost/46723] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 06:00:14,294 INFO [Listener at localhost/46723] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 06:00:14,299 INFO [Listener at localhost/46723] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/java.io.tmpdir/Jetty_localhost_40571_datanode____3e1gl6/webapp 2023-09-04 06:00:14,393 INFO [Listener at localhost/46723] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40571 2023-09-04 06:00:14,406 WARN [Listener at localhost/36343] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 06:00:14,411 WARN [AsyncFSWAL-0-hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914-prefix:jenkins-hbase4.apache.org,41993,1693807200675] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:14,412 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C41993%2C1693807200675:(num 1693807202127) roll requested 2023-09-04 06:00:14,431 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41647,DS-d3eadf9a-06ef-4066-97c7-62deb958d9bc,DISK] 2023-09-04 06:00:14,431 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35711,DS-770909ef-a6eb-46a2-a45d-541691e80c67,DISK] 2023-09-04 06:00:14,435 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.1693807202127 with entries=6, filesize=3.19 KB; new WAL /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.1693807214412 2023-09-04 06:00:14,437 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41647,DS-d3eadf9a-06ef-4066-97c7-62deb958d9bc,DISK], DatanodeInfoWithStorage[127.0.0.1:35711,DS-770909ef-a6eb-46a2-a45d-541691e80c67,DISK]] 2023-09-04 06:00:14,437 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.1693807202127 is not closed yet, will try archiving it next time 2023-09-04 06:00:14,438 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-09-04 06:00:14,438 INFO [Listener at localhost/36343] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-09-04 06:00:14,439 DEBUG [Listener at localhost/36343] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0f1dfe88 to 127.0.0.1:55230 2023-09-04 06:00:14,440 DEBUG [Listener at localhost/36343] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:00:14,440 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:14,440 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:14,440 DEBUG [Listener at localhost/36343] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-09-04 06:00:14,441 DEBUG [Listener at localhost/36343] util.JVMClusterUtil(257): Found active master hash=1649127331, stopped=false 2023-09-04 06:00:14,441 INFO [Listener at localhost/36343] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,37663,1693807199468 2023-09-04 06:00:14,442 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.1693807202127 2023-09-04 06:00:14,446 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): regionserver:41993-0x1007585f92b0001, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 06:00:14,446 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 06:00:14,446 INFO [Listener at localhost/36343] procedure2.ProcedureExecutor(628): Stopping 2023-09-04 06:00:14,446 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:14,447 DEBUG [Listener at localhost/36343] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x580f8e8b to 127.0.0.1:55230 2023-09-04 06:00:14,448 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41993-0x1007585f92b0001, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:00:14,448 DEBUG [Listener at localhost/36343] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:00:14,448 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:00:14,448 INFO [Listener at localhost/36343] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41993,1693807200675' ***** 2023-09-04 06:00:14,448 WARN [IPC Server handler 3 on default port 42011] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.1693807202127 has not been closed. Lease recovery is in progress. RecoveryId = 1015 for block blk_1073741832_1008 2023-09-04 06:00:14,448 INFO [Listener at localhost/36343] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-09-04 06:00:14,448 INFO [RS:0;jenkins-hbase4:41993] regionserver.HeapMemoryManager(220): Stopping 2023-09-04 06:00:14,449 INFO [RS:0;jenkins-hbase4:41993] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-09-04 06:00:14,449 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-09-04 06:00:14,449 INFO [RS:0;jenkins-hbase4:41993] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-09-04 06:00:14,449 INFO [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(3305): Received CLOSE for 109b79e843fc9fd6517ada18445d914e 2023-09-04 06:00:14,450 INFO [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(3305): Received CLOSE for 4486abc3f5c29833f5dc8026cdf7364e 2023-09-04 06:00:14,451 INFO [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41993,1693807200675 2023-09-04 06:00:14,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 109b79e843fc9fd6517ada18445d914e, disabling compactions & flushes 2023-09-04 06:00:14,451 DEBUG [RS:0;jenkins-hbase4:41993] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x652a9a0c to 127.0.0.1:55230 2023-09-04 06:00:14,451 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e. 2023-09-04 06:00:14,451 DEBUG [RS:0;jenkins-hbase4:41993] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:00:14,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e. 2023-09-04 06:00:14,452 INFO [RS:0;jenkins-hbase4:41993] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-09-04 06:00:14,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e. after waiting 0 ms 2023-09-04 06:00:14,452 INFO [RS:0;jenkins-hbase4:41993] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-09-04 06:00:14,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e. 2023-09-04 06:00:14,452 INFO [RS:0;jenkins-hbase4:41993] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-09-04 06:00:14,452 INFO [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-09-04 06:00:14,452 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.1693807202127 after 10ms 2023-09-04 06:00:14,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 109b79e843fc9fd6517ada18445d914e 1/1 column families, dataSize=2.05 KB heapSize=2.45 KB 2023-09-04 06:00:14,453 INFO [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-09-04 06:00:14,453 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 06:00:14,453 DEBUG [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(1478): Online Regions={109b79e843fc9fd6517ada18445d914e=TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e., 4486abc3f5c29833f5dc8026cdf7364e=hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e., 1588230740=hbase:meta,,1.1588230740} 2023-09-04 06:00:14,454 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 06:00:14,454 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 06:00:14,454 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 06:00:14,454 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 06:00:14,454 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.92 KB heapSize=5.45 KB 2023-09-04 06:00:14,455 DEBUG [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(1504): Waiting on 109b79e843fc9fd6517ada18445d914e, 1588230740, 4486abc3f5c29833f5dc8026cdf7364e 2023-09-04 06:00:14,464 WARN [AsyncFSWAL-0-hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914-prefix:jenkins-hbase4.apache.org,41993,1693807200675.meta] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:14,465 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C41993%2C1693807200675.meta:.meta(num 1693807202248) roll requested 2023-09-04 06:00:14,484 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41647,DS-d3eadf9a-06ef-4066-97c7-62deb958d9bc,DISK] 2023-09-04 06:00:14,484 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44339,DS-e6b2c7c2-53e1-4219-8045-0839a0a827ba,DISK] 2023-09-04 06:00:14,494 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.meta.1693807202248.meta with entries=12, filesize=3.87 KB; new WAL /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.meta.1693807214465.meta 2023-09-04 06:00:14,495 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44339,DS-e6b2c7c2-53e1-4219-8045-0839a0a827ba,DISK], DatanodeInfoWithStorage[127.0.0.1:41647,DS-d3eadf9a-06ef-4066-97c7-62deb958d9bc,DISK]] 2023-09-04 06:00:14,495 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.meta.1693807202248.meta is not closed yet, will try archiving it next time 2023-09-04 06:00:14,495 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:14,497 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:14,498 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.meta.1693807202248.meta 2023-09-04 06:00:14,499 WARN [IPC Server handler 1 on default port 42011] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.meta.1693807202248.meta has not been closed. Lease recovery is in progress. RecoveryId = 1017 for block blk_1073741833_1009 2023-09-04 06:00:14,500 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.meta.1693807202248.meta after 2ms 2023-09-04 06:00:14,546 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb2ed1bd9b373bbe0: Processing first storage report for DS-e4341291-06c9-49f0-b7b0-cb6586ad625f from datanode 32d93fbe-e3a0-4b19-a3c3-6c2b751b4e0f 2023-09-04 06:00:14,548 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb2ed1bd9b373bbe0: from storage DS-e4341291-06c9-49f0-b7b0-cb6586ad625f node DatanodeRegistration(127.0.0.1:40947, datanodeUuid=32d93fbe-e3a0-4b19-a3c3-6c2b751b4e0f, infoPort=33525, infoSecurePort=0, ipcPort=36343, storageInfo=lv=-57;cid=testClusterID;nsid=321774033;c=1693807196473), blocks: 6, hasStaleStorage: false, processing time: 3 msecs, invalidatedBlocks: 0 2023-09-04 06:00:14,549 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb2ed1bd9b373bbe0: Processing first storage report for DS-582b401c-1923-4de2-af87-0a1ca7da5529 from datanode 32d93fbe-e3a0-4b19-a3c3-6c2b751b4e0f 2023-09-04 06:00:14,549 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb2ed1bd9b373bbe0: from storage DS-582b401c-1923-4de2-af87-0a1ca7da5529 node DatanodeRegistration(127.0.0.1:40947, datanodeUuid=32d93fbe-e3a0-4b19-a3c3-6c2b751b4e0f, infoPort=33525, infoSecurePort=0, ipcPort=36343, storageInfo=lv=-57;cid=testClusterID;nsid=321774033;c=1693807196473), blocks: 7, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:00:14,551 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.69 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/.tmp/info/dc16b4ea697341c08c48a9fd428ed2a3 2023-09-04 06:00:14,552 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.05 KB at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/default/TestLogRolling-testLogRollOnDatanodeDeath/109b79e843fc9fd6517ada18445d914e/.tmp/info/598a26831f494e8ca0f6984ec659de85 2023-09-04 06:00:14,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/default/TestLogRolling-testLogRollOnDatanodeDeath/109b79e843fc9fd6517ada18445d914e/.tmp/info/598a26831f494e8ca0f6984ec659de85 as hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/default/TestLogRolling-testLogRollOnDatanodeDeath/109b79e843fc9fd6517ada18445d914e/info/598a26831f494e8ca0f6984ec659de85 2023-09-04 06:00:14,619 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/default/TestLogRolling-testLogRollOnDatanodeDeath/109b79e843fc9fd6517ada18445d914e/info/598a26831f494e8ca0f6984ec659de85, entries=2, sequenceid=6, filesize=6.8 K 2023-09-04 06:00:14,621 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/.tmp/table/0465fcdf22f24079ba2bdae58a661007 2023-09-04 06:00:14,623 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.05 KB/2102, heapSize ~2.44 KB/2496, currentSize=0 B/0 for 109b79e843fc9fd6517ada18445d914e in 170ms, sequenceid=6, compaction requested=false 2023-09-04 06:00:14,634 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/default/TestLogRolling-testLogRollOnDatanodeDeath/109b79e843fc9fd6517ada18445d914e/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-09-04 06:00:14,635 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/.tmp/info/dc16b4ea697341c08c48a9fd428ed2a3 as hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/info/dc16b4ea697341c08c48a9fd428ed2a3 2023-09-04 06:00:14,635 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e. 2023-09-04 06:00:14,635 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 109b79e843fc9fd6517ada18445d914e: 2023-09-04 06:00:14,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1693807204121.109b79e843fc9fd6517ada18445d914e. 2023-09-04 06:00:14,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4486abc3f5c29833f5dc8026cdf7364e, disabling compactions & flushes 2023-09-04 06:00:14,636 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e. 2023-09-04 06:00:14,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e. 2023-09-04 06:00:14,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e. after waiting 0 ms 2023-09-04 06:00:14,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e. 2023-09-04 06:00:14,636 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 4486abc3f5c29833f5dc8026cdf7364e 1/1 column families, dataSize=78 B heapSize=488 B 2023-09-04 06:00:14,646 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/info/dc16b4ea697341c08c48a9fd428ed2a3, entries=20, sequenceid=14, filesize=7.4 K 2023-09-04 06:00:14,648 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/.tmp/table/0465fcdf22f24079ba2bdae58a661007 as hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/table/0465fcdf22f24079ba2bdae58a661007 2023-09-04 06:00:14,656 DEBUG [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(1504): Waiting on 1588230740, 4486abc3f5c29833f5dc8026cdf7364e 2023-09-04 06:00:14,657 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/namespace/4486abc3f5c29833f5dc8026cdf7364e/.tmp/info/c2446aab2da14ccda5fb8602cb495b3f 2023-09-04 06:00:14,662 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/table/0465fcdf22f24079ba2bdae58a661007, entries=4, sequenceid=14, filesize=4.8 K 2023-09-04 06:00:14,664 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.92 KB/2994, heapSize ~5.16 KB/5288, currentSize=0 B/0 for 1588230740 in 209ms, sequenceid=14, compaction requested=false 2023-09-04 06:00:14,670 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/namespace/4486abc3f5c29833f5dc8026cdf7364e/.tmp/info/c2446aab2da14ccda5fb8602cb495b3f as hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/namespace/4486abc3f5c29833f5dc8026cdf7364e/info/c2446aab2da14ccda5fb8602cb495b3f 2023-09-04 06:00:14,673 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-09-04 06:00:14,674 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-04 06:00:14,675 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 06:00:14,675 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 06:00:14,675 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-09-04 06:00:14,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/namespace/4486abc3f5c29833f5dc8026cdf7364e/info/c2446aab2da14ccda5fb8602cb495b3f, entries=2, sequenceid=6, filesize=4.8 K 2023-09-04 06:00:14,682 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 4486abc3f5c29833f5dc8026cdf7364e in 46ms, sequenceid=6, compaction requested=false 2023-09-04 06:00:14,691 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/data/hbase/namespace/4486abc3f5c29833f5dc8026cdf7364e/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-09-04 06:00:14,692 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e. 2023-09-04 06:00:14,693 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4486abc3f5c29833f5dc8026cdf7364e: 2023-09-04 06:00:14,693 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693807202516.4486abc3f5c29833f5dc8026cdf7364e. 2023-09-04 06:00:14,856 INFO [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41993,1693807200675; all regions closed. 2023-09-04 06:00:14,975 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-09-04 06:00:14,975 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-09-04 06:00:15,977 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-09-04 06:00:18,454 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.1693807202127 after 4012ms 2023-09-04 06:00:18,455 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.1693807202127 to hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/oldWALs/jenkins-hbase4.apache.org%2C41993%2C1693807200675.1693807202127 2023-09-04 06:00:18,501 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.meta.1693807202248.meta after 4003ms 2023-09-04 06:00:18,502 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/WALs/jenkins-hbase4.apache.org,41993,1693807200675/jenkins-hbase4.apache.org%2C41993%2C1693807200675.meta.1693807202248.meta to hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/oldWALs/jenkins-hbase4.apache.org%2C41993%2C1693807200675.meta.1693807202248.meta 2023-09-04 06:00:18,507 DEBUG [RS:0;jenkins-hbase4:41993] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/oldWALs 2023-09-04 06:00:18,508 INFO [RS:0;jenkins-hbase4:41993] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41993%2C1693807200675.meta:.meta(num 1693807214465) 2023-09-04 06:00:18,517 DEBUG [RS:0;jenkins-hbase4:41993] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/oldWALs 2023-09-04 06:00:18,517 INFO [RS:0;jenkins-hbase4:41993] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41993%2C1693807200675:(num 1693807214412) 2023-09-04 06:00:18,517 DEBUG [RS:0;jenkins-hbase4:41993] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:00:18,517 INFO [RS:0;jenkins-hbase4:41993] regionserver.LeaseManager(133): Closed leases 2023-09-04 06:00:18,517 INFO [RS:0;jenkins-hbase4:41993] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-09-04 06:00:18,517 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 06:00:18,518 INFO [RS:0;jenkins-hbase4:41993] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41993 2023-09-04 06:00:18,525 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): regionserver:41993-0x1007585f92b0001, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41993,1693807200675 2023-09-04 06:00:18,525 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 06:00:18,525 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): regionserver:41993-0x1007585f92b0001, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 06:00:18,526 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41993,1693807200675] 2023-09-04 06:00:18,526 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41993,1693807200675; numProcessing=1 2023-09-04 06:00:18,528 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41993,1693807200675 already deleted, retry=false 2023-09-04 06:00:18,528 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41993,1693807200675 expired; onlineServers=0 2023-09-04 06:00:18,528 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,37663,1693807199468' ***** 2023-09-04 06:00:18,528 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-09-04 06:00:18,528 DEBUG [M:0;jenkins-hbase4:37663] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2376a598, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 06:00:18,529 INFO [M:0;jenkins-hbase4:37663] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,37663,1693807199468 2023-09-04 06:00:18,529 INFO [M:0;jenkins-hbase4:37663] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,37663,1693807199468; all regions closed. 2023-09-04 06:00:18,529 DEBUG [M:0;jenkins-hbase4:37663] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:00:18,529 DEBUG [M:0;jenkins-hbase4:37663] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-09-04 06:00:18,529 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-09-04 06:00:18,530 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693807201768] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693807201768,5,FailOnTimeoutGroup] 2023-09-04 06:00:18,530 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693807201768] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693807201768,5,FailOnTimeoutGroup] 2023-09-04 06:00:18,530 DEBUG [M:0;jenkins-hbase4:37663] cleaner.HFileCleaner(317): Stopping file delete threads 2023-09-04 06:00:18,531 INFO [M:0;jenkins-hbase4:37663] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-09-04 06:00:18,531 INFO [M:0;jenkins-hbase4:37663] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-09-04 06:00:18,531 INFO [M:0;jenkins-hbase4:37663] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-09-04 06:00:18,532 DEBUG [M:0;jenkins-hbase4:37663] master.HMaster(1512): Stopping service threads 2023-09-04 06:00:18,532 INFO [M:0;jenkins-hbase4:37663] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-09-04 06:00:18,532 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-09-04 06:00:18,533 ERROR [M:0;jenkins-hbase4:37663] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-09-04 06:00:18,533 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] 2023-09-04 06:00:18,533 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 06:00:18,533 INFO [M:0;jenkins-hbase4:37663] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-09-04 06:00:18,533 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-09-04 06:00:18,534 DEBUG [M:0;jenkins-hbase4:37663] zookeeper.ZKUtil(398): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-09-04 06:00:18,534 WARN [M:0;jenkins-hbase4:37663] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-09-04 06:00:18,534 INFO [M:0;jenkins-hbase4:37663] assignment.AssignmentManager(315): Stopping assignment manager 2023-09-04 06:00:18,534 INFO [M:0;jenkins-hbase4:37663] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-09-04 06:00:18,535 DEBUG [M:0;jenkins-hbase4:37663] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 06:00:18,535 INFO [M:0;jenkins-hbase4:37663] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:00:18,535 DEBUG [M:0;jenkins-hbase4:37663] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:00:18,535 DEBUG [M:0;jenkins-hbase4:37663] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 06:00:18,535 DEBUG [M:0;jenkins-hbase4:37663] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:00:18,535 INFO [M:0;jenkins-hbase4:37663] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.08 KB heapSize=45.73 KB 2023-09-04 06:00:18,536 WARN [AsyncFSWAL-0-hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData-prefix:jenkins-hbase4.apache.org,37663,1693807199468] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:18,536 DEBUG [master:store-WAL-Roller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C37663%2C1693807199468:(num 1693807201120) roll requested 2023-09-04 06:00:18,552 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40947,DS-e4341291-06c9-49f0-b7b0-cb6586ad625f,DISK] 2023-09-04 06:00:18,553 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44339,DS-e6b2c7c2-53e1-4219-8045-0839a0a827ba,DISK] 2023-09-04 06:00:18,557 INFO [master:store-WAL-Roller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/WALs/jenkins-hbase4.apache.org,37663,1693807199468/jenkins-hbase4.apache.org%2C37663%2C1693807199468.1693807201120 with entries=89, filesize=43.93 KB; new WAL /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/WALs/jenkins-hbase4.apache.org,37663,1693807199468/jenkins-hbase4.apache.org%2C37663%2C1693807199468.1693807218536 2023-09-04 06:00:18,558 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40947,DS-e4341291-06c9-49f0-b7b0-cb6586ad625f,DISK], DatanodeInfoWithStorage[127.0.0.1:44339,DS-e6b2c7c2-53e1-4219-8045-0839a0a827ba,DISK]] 2023-09-04 06:00:18,558 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(716): hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/WALs/jenkins-hbase4.apache.org,37663,1693807199468/jenkins-hbase4.apache.org%2C37663%2C1693807199468.1693807201120 is not closed yet, will try archiving it next time 2023-09-04 06:00:18,558 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:18,558 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 06:00:18,558 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/WALs/jenkins-hbase4.apache.org,37663,1693807199468/jenkins-hbase4.apache.org%2C37663%2C1693807199468.1693807201120 2023-09-04 06:00:18,559 WARN [IPC Server handler 4 on default port 42011] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/WALs/jenkins-hbase4.apache.org,37663,1693807199468/jenkins-hbase4.apache.org%2C37663%2C1693807199468.1693807201120 has not been closed. Lease recovery is in progress. RecoveryId = 1023 for block blk_1073741829_1005 2023-09-04 06:00:18,560 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/WALs/jenkins-hbase4.apache.org,37663,1693807199468/jenkins-hbase4.apache.org%2C37663%2C1693807199468.1693807201120 after 2ms 2023-09-04 06:00:18,575 INFO [M:0;jenkins-hbase4:37663] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.08 KB at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/1fd1f52fa9104c1880911583727d430c 2023-09-04 06:00:18,583 DEBUG [M:0;jenkins-hbase4:37663] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/1fd1f52fa9104c1880911583727d430c as hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/1fd1f52fa9104c1880911583727d430c 2023-09-04 06:00:18,590 INFO [M:0;jenkins-hbase4:37663] regionserver.HStore(1080): Added hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/1fd1f52fa9104c1880911583727d430c, entries=11, sequenceid=91, filesize=7.0 K 2023-09-04 06:00:18,591 INFO [M:0;jenkins-hbase4:37663] regionserver.HRegion(2948): Finished flush of dataSize ~38.08 KB/38997, heapSize ~45.72 KB/46816, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 56ms, sequenceid=91, compaction requested=false 2023-09-04 06:00:18,593 INFO [M:0;jenkins-hbase4:37663] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:00:18,593 DEBUG [M:0;jenkins-hbase4:37663] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 06:00:18,627 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): regionserver:41993-0x1007585f92b0001, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:00:18,627 INFO [RS:0;jenkins-hbase4:41993] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41993,1693807200675; zookeeper connection closed. 2023-09-04 06:00:18,627 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): regionserver:41993-0x1007585f92b0001, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:00:18,628 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@54b927f8] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@54b927f8 2023-09-04 06:00:18,628 INFO [Listener at localhost/36343] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-09-04 06:00:20,170 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-04 06:00:20,206 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-04 06:00:22,561 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/WALs/jenkins-hbase4.apache.org,37663,1693807199468/jenkins-hbase4.apache.org%2C37663%2C1693807199468.1693807201120 after 4003ms 2023-09-04 06:00:22,561 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/WALs/jenkins-hbase4.apache.org,37663,1693807199468/jenkins-hbase4.apache.org%2C37663%2C1693807199468.1693807201120 to hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/oldWALs/jenkins-hbase4.apache.org%2C37663%2C1693807199468.1693807201120 2023-09-04 06:00:22,568 INFO [WAL-Archive-0] region.MasterRegionUtils(50): Moved hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/MasterData/oldWALs/jenkins-hbase4.apache.org%2C37663%2C1693807199468.1693807201120 to hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/oldWALs/jenkins-hbase4.apache.org%2C37663%2C1693807199468.1693807201120$masterlocalwal$ 2023-09-04 06:00:22,568 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 06:00:22,568 INFO [M:0;jenkins-hbase4:37663] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-09-04 06:00:22,568 INFO [M:0;jenkins-hbase4:37663] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37663 2023-09-04 06:00:22,572 DEBUG [M:0;jenkins-hbase4:37663] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,37663,1693807199468 already deleted, retry=false 2023-09-04 06:00:22,674 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:00:22,674 INFO [M:0;jenkins-hbase4:37663] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,37663,1693807199468; zookeeper connection closed. 2023-09-04 06:00:22,674 DEBUG [Listener at localhost/44875-EventThread] zookeeper.ZKWatcher(600): master:37663-0x1007585f92b0000, quorum=127.0.0.1:55230, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:00:22,676 WARN [Listener at localhost/36343] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 06:00:22,679 INFO [Listener at localhost/36343] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:00:22,783 WARN [BP-1630235210-172.31.14.131-1693807196473 heartbeating to localhost/127.0.0.1:42011] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 06:00:22,783 WARN [BP-1630235210-172.31.14.131-1693807196473 heartbeating to localhost/127.0.0.1:42011] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1630235210-172.31.14.131-1693807196473 (Datanode Uuid 32d93fbe-e3a0-4b19-a3c3-6c2b751b4e0f) service to localhost/127.0.0.1:42011 2023-09-04 06:00:22,784 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a/dfs/data/data3/current/BP-1630235210-172.31.14.131-1693807196473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:00:22,784 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a/dfs/data/data4/current/BP-1630235210-172.31.14.131-1693807196473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:00:22,787 WARN [Listener at localhost/36343] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 06:00:22,792 INFO [Listener at localhost/36343] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:00:22,896 WARN [BP-1630235210-172.31.14.131-1693807196473 heartbeating to localhost/127.0.0.1:42011] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 06:00:22,896 WARN [BP-1630235210-172.31.14.131-1693807196473 heartbeating to localhost/127.0.0.1:42011] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1630235210-172.31.14.131-1693807196473 (Datanode Uuid d73cee86-9872-478e-b72f-008a0fbced21) service to localhost/127.0.0.1:42011 2023-09-04 06:00:22,896 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a/dfs/data/data9/current/BP-1630235210-172.31.14.131-1693807196473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:00:22,897 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a/dfs/data/data10/current/BP-1630235210-172.31.14.131-1693807196473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:00:22,898 WARN [Listener at localhost/36343] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 06:00:22,901 INFO [Listener at localhost/36343] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:00:23,005 WARN [BP-1630235210-172.31.14.131-1693807196473 heartbeating to localhost/127.0.0.1:42011] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 06:00:23,005 WARN [BP-1630235210-172.31.14.131-1693807196473 heartbeating to localhost/127.0.0.1:42011] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1630235210-172.31.14.131-1693807196473 (Datanode Uuid 09aefbf6-d44c-4fb8-b8b7-a521c07d3e76) service to localhost/127.0.0.1:42011 2023-09-04 06:00:23,005 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a/dfs/data/data7/current/BP-1630235210-172.31.14.131-1693807196473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:00:23,006 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a/dfs/data/data8/current/BP-1630235210-172.31.14.131-1693807196473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:00:23,007 WARN [Listener at localhost/36343] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 06:00:23,010 INFO [Listener at localhost/36343] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:00:23,115 WARN [BP-1630235210-172.31.14.131-1693807196473 heartbeating to localhost/127.0.0.1:42011] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 06:00:23,116 WARN [BP-1630235210-172.31.14.131-1693807196473 heartbeating to localhost/127.0.0.1:42011] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1630235210-172.31.14.131-1693807196473 (Datanode Uuid 01f55e7b-32c7-4705-b9e7-70560f25f10e) service to localhost/127.0.0.1:42011 2023-09-04 06:00:23,116 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a/dfs/data/data5/current/BP-1630235210-172.31.14.131-1693807196473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:00:23,117 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a/dfs/data/data6/current/BP-1630235210-172.31.14.131-1693807196473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:00:23,118 WARN [Listener at localhost/36343] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 06:00:23,123 INFO [Listener at localhost/36343] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:00:23,224 WARN [BP-1630235210-172.31.14.131-1693807196473 heartbeating to localhost/127.0.0.1:42011] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 06:00:23,224 WARN [BP-1630235210-172.31.14.131-1693807196473 heartbeating to localhost/127.0.0.1:42011] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1630235210-172.31.14.131-1693807196473 (Datanode Uuid b6023de8-7e06-4f53-a488-ff54e50d0cdd) service to localhost/127.0.0.1:42011 2023-09-04 06:00:23,225 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a/dfs/data/data1/current/BP-1630235210-172.31.14.131-1693807196473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:00:23,225 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/cluster_a0506aee-6201-fcec-f669-a54c02c2461a/dfs/data/data2/current/BP-1630235210-172.31.14.131-1693807196473] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:00:23,259 INFO [Listener at localhost/36343] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:00:23,380 INFO [Listener at localhost/36343] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-09-04 06:00:23,428 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-09-04 06:00:23,442 INFO [Listener at localhost/36343] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=78 (was 10) Potentially hanging thread: nioEventLoopGroup-2-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (694220413) connection to localhost/127.0.0.1:42011 from jenkins.hfs.0 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (694220413) connection to localhost/127.0.0.1:42011 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/36343 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (694220413) connection to localhost/127.0.0.1:42011 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:42011 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: HFileArchiver-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-2-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@6262a4e9 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-3-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:42011 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) - Thread LEAK? -, OpenFileDescriptor=467 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=144 (was 165), ProcessCount=166 (was 166), AvailableMemoryMB=8493 (was 9236) 2023-09-04 06:00:23,453 INFO [Listener at localhost/36343] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=78, OpenFileDescriptor=467, MaxFileDescriptor=60000, SystemLoadAverage=144, ProcessCount=166, AvailableMemoryMB=8493 2023-09-04 06:00:23,453 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-09-04 06:00:23,454 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/hadoop.log.dir so I do NOT create it in target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5 2023-09-04 06:00:23,454 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e7bfeecd-56b2-2635-3d5e-11a32a550ef1/hadoop.tmp.dir so I do NOT create it in target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5 2023-09-04 06:00:23,454 INFO [Listener at localhost/36343] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/cluster_406daa38-4a7a-08be-d885-7f6397132708, deleteOnExit=true 2023-09-04 06:00:23,454 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-09-04 06:00:23,454 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/test.cache.data in system properties and HBase conf 2023-09-04 06:00:23,455 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/hadoop.tmp.dir in system properties and HBase conf 2023-09-04 06:00:23,455 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/hadoop.log.dir in system properties and HBase conf 2023-09-04 06:00:23,455 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/mapreduce.cluster.local.dir in system properties and HBase conf 2023-09-04 06:00:23,455 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-09-04 06:00:23,455 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-09-04 06:00:23,455 DEBUG [Listener at localhost/36343] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-09-04 06:00:23,456 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-09-04 06:00:23,456 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-09-04 06:00:23,456 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-09-04 06:00:23,456 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 06:00:23,456 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-09-04 06:00:23,457 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-09-04 06:00:23,457 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 06:00:23,457 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 06:00:23,457 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-09-04 06:00:23,457 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/nfs.dump.dir in system properties and HBase conf 2023-09-04 06:00:23,457 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/java.io.tmpdir in system properties and HBase conf 2023-09-04 06:00:23,457 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 06:00:23,457 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-09-04 06:00:23,458 INFO [Listener at localhost/36343] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-09-04 06:00:23,463 WARN [Listener at localhost/36343] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 06:00:23,463 WARN [Listener at localhost/36343] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 06:00:23,514 WARN [Listener at localhost/36343] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 06:00:23,516 INFO [Listener at localhost/36343] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 06:00:23,521 INFO [Listener at localhost/36343] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/java.io.tmpdir/Jetty_localhost_39409_hdfs____.z5oain/webapp 2023-09-04 06:00:23,616 INFO [Listener at localhost/36343] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39409 2023-09-04 06:00:23,622 WARN [Listener at localhost/36343] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 06:00:23,622 WARN [Listener at localhost/36343] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 06:00:23,674 WARN [Listener at localhost/43627] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 06:00:23,687 WARN [Listener at localhost/43627] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 06:00:23,690 WARN [Listener at localhost/43627] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 06:00:23,692 INFO [Listener at localhost/43627] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 06:00:23,697 INFO [Listener at localhost/43627] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/java.io.tmpdir/Jetty_localhost_42951_datanode____vx8dz2/webapp 2023-09-04 06:00:23,788 INFO [Listener at localhost/43627] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42951 2023-09-04 06:00:23,796 WARN [Listener at localhost/43437] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 06:00:23,809 WARN [Listener at localhost/43437] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 06:00:23,811 WARN [Listener at localhost/43437] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 06:00:23,812 INFO [Listener at localhost/43437] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 06:00:23,815 INFO [Listener at localhost/43437] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/java.io.tmpdir/Jetty_localhost_44307_datanode____k72kf5/webapp 2023-09-04 06:00:23,905 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb68cb0811f01548e: Processing first storage report for DS-1f884a02-e206-4857-ac2b-56c1647f3b10 from datanode e7691a50-e56f-4eac-bd8e-c4481fb69549 2023-09-04 06:00:23,906 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb68cb0811f01548e: from storage DS-1f884a02-e206-4857-ac2b-56c1647f3b10 node DatanodeRegistration(127.0.0.1:35561, datanodeUuid=e7691a50-e56f-4eac-bd8e-c4481fb69549, infoPort=44277, infoSecurePort=0, ipcPort=43437, storageInfo=lv=-57;cid=testClusterID;nsid=1637306855;c=1693807223466), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:00:23,906 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb68cb0811f01548e: Processing first storage report for DS-5d62778c-7327-4647-b459-48592e9a53c3 from datanode e7691a50-e56f-4eac-bd8e-c4481fb69549 2023-09-04 06:00:23,906 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb68cb0811f01548e: from storage DS-5d62778c-7327-4647-b459-48592e9a53c3 node DatanodeRegistration(127.0.0.1:35561, datanodeUuid=e7691a50-e56f-4eac-bd8e-c4481fb69549, infoPort=44277, infoSecurePort=0, ipcPort=43437, storageInfo=lv=-57;cid=testClusterID;nsid=1637306855;c=1693807223466), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:00:23,915 INFO [Listener at localhost/43437] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44307 2023-09-04 06:00:23,923 WARN [Listener at localhost/35025] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 06:00:24,023 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x7575123daec6b14a: Processing first storage report for DS-015b0936-ef74-470f-9c98-0f7973192b02 from datanode e33e49c9-c2d4-43ae-9334-4f1ff4cfb445 2023-09-04 06:00:24,023 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x7575123daec6b14a: from storage DS-015b0936-ef74-470f-9c98-0f7973192b02 node DatanodeRegistration(127.0.0.1:46443, datanodeUuid=e33e49c9-c2d4-43ae-9334-4f1ff4cfb445, infoPort=45797, infoSecurePort=0, ipcPort=35025, storageInfo=lv=-57;cid=testClusterID;nsid=1637306855;c=1693807223466), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:00:24,023 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x7575123daec6b14a: Processing first storage report for DS-b02f08e0-aa6a-4ebb-9473-2721c2b59c48 from datanode e33e49c9-c2d4-43ae-9334-4f1ff4cfb445 2023-09-04 06:00:24,023 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x7575123daec6b14a: from storage DS-b02f08e0-aa6a-4ebb-9473-2721c2b59c48 node DatanodeRegistration(127.0.0.1:46443, datanodeUuid=e33e49c9-c2d4-43ae-9334-4f1ff4cfb445, infoPort=45797, infoSecurePort=0, ipcPort=35025, storageInfo=lv=-57;cid=testClusterID;nsid=1637306855;c=1693807223466), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:00:24,030 DEBUG [Listener at localhost/35025] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5 2023-09-04 06:00:24,033 INFO [Listener at localhost/35025] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/cluster_406daa38-4a7a-08be-d885-7f6397132708/zookeeper_0, clientPort=49156, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/cluster_406daa38-4a7a-08be-d885-7f6397132708/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/cluster_406daa38-4a7a-08be-d885-7f6397132708/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-09-04 06:00:24,035 INFO [Listener at localhost/35025] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=49156 2023-09-04 06:00:24,035 INFO [Listener at localhost/35025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:00:24,036 INFO [Listener at localhost/35025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:00:24,052 INFO [Listener at localhost/35025] util.FSUtils(471): Created version file at hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3 with version=8 2023-09-04 06:00:24,052 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/hbase-staging 2023-09-04 06:00:24,054 INFO [Listener at localhost/35025] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 06:00:24,055 INFO [Listener at localhost/35025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:00:24,055 INFO [Listener at localhost/35025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 06:00:24,055 INFO [Listener at localhost/35025] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 06:00:24,055 INFO [Listener at localhost/35025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:00:24,055 INFO [Listener at localhost/35025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 06:00:24,055 INFO [Listener at localhost/35025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 06:00:24,056 INFO [Listener at localhost/35025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38737 2023-09-04 06:00:24,057 INFO [Listener at localhost/35025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:00:24,058 INFO [Listener at localhost/35025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:00:24,059 INFO [Listener at localhost/35025] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38737 connecting to ZooKeeper ensemble=127.0.0.1:49156 2023-09-04 06:00:24,066 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:387370x0, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 06:00:24,067 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:38737-0x10075865c870000 connected 2023-09-04 06:00:24,086 DEBUG [Listener at localhost/35025] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 06:00:24,087 DEBUG [Listener at localhost/35025] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:00:24,087 DEBUG [Listener at localhost/35025] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 06:00:24,088 DEBUG [Listener at localhost/35025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38737 2023-09-04 06:00:24,088 DEBUG [Listener at localhost/35025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38737 2023-09-04 06:00:24,088 DEBUG [Listener at localhost/35025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38737 2023-09-04 06:00:24,088 DEBUG [Listener at localhost/35025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38737 2023-09-04 06:00:24,088 DEBUG [Listener at localhost/35025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38737 2023-09-04 06:00:24,088 INFO [Listener at localhost/35025] master.HMaster(444): hbase.rootdir=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3, hbase.cluster.distributed=false 2023-09-04 06:00:24,103 INFO [Listener at localhost/35025] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 06:00:24,103 INFO [Listener at localhost/35025] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:00:24,103 INFO [Listener at localhost/35025] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 06:00:24,103 INFO [Listener at localhost/35025] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 06:00:24,103 INFO [Listener at localhost/35025] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:00:24,103 INFO [Listener at localhost/35025] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 06:00:24,104 INFO [Listener at localhost/35025] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 06:00:24,107 INFO [Listener at localhost/35025] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37843 2023-09-04 06:00:24,108 INFO [Listener at localhost/35025] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-04 06:00:24,110 DEBUG [Listener at localhost/35025] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-09-04 06:00:24,111 INFO [Listener at localhost/35025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:00:24,112 INFO [Listener at localhost/35025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:00:24,113 INFO [Listener at localhost/35025] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37843 connecting to ZooKeeper ensemble=127.0.0.1:49156 2023-09-04 06:00:24,124 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:378430x0, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 06:00:24,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:37843-0x10075865c870001 connected 2023-09-04 06:00:24,125 DEBUG [Listener at localhost/35025] zookeeper.ZKUtil(164): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 06:00:24,126 DEBUG [Listener at localhost/35025] zookeeper.ZKUtil(164): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:00:24,127 DEBUG [Listener at localhost/35025] zookeeper.ZKUtil(164): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 06:00:24,129 DEBUG [Listener at localhost/35025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37843 2023-09-04 06:00:24,130 DEBUG [Listener at localhost/35025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37843 2023-09-04 06:00:24,133 DEBUG [Listener at localhost/35025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37843 2023-09-04 06:00:24,137 DEBUG [Listener at localhost/35025] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37843 2023-09-04 06:00:24,138 DEBUG [Listener at localhost/35025] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37843 2023-09-04 06:00:24,140 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,38737,1693807224054 2023-09-04 06:00:24,142 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 06:00:24,143 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,38737,1693807224054 2023-09-04 06:00:24,146 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 06:00:24,146 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 06:00:24,146 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:24,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 06:00:24,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 06:00:24,147 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,38737,1693807224054 from backup master directory 2023-09-04 06:00:24,149 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,38737,1693807224054 2023-09-04 06:00:24,149 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 06:00:24,149 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 06:00:24,149 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,38737,1693807224054 2023-09-04 06:00:24,178 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/hbase.id with ID: 86709527-0204-4306-9ec8-06aa3e217791 2023-09-04 06:00:24,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:00:24,198 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:24,210 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x7d2b5906 to 127.0.0.1:49156 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 06:00:24,216 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@fb5e4dd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 06:00:24,216 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-04 06:00:24,217 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-09-04 06:00:24,217 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 06:00:24,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/MasterData/data/master/store-tmp 2023-09-04 06:00:24,232 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:24,232 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 06:00:24,232 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:00:24,232 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:00:24,232 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 06:00:24,232 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:00:24,232 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:00:24,232 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 06:00:24,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/MasterData/WALs/jenkins-hbase4.apache.org,38737,1693807224054 2023-09-04 06:00:24,237 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38737%2C1693807224054, suffix=, logDir=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/MasterData/WALs/jenkins-hbase4.apache.org,38737,1693807224054, archiveDir=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/MasterData/oldWALs, maxLogs=10 2023-09-04 06:00:24,261 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35561,DS-1f884a02-e206-4857-ac2b-56c1647f3b10,DISK] 2023-09-04 06:00:24,263 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46443,DS-015b0936-ef74-470f-9c98-0f7973192b02,DISK] 2023-09-04 06:00:24,269 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/MasterData/WALs/jenkins-hbase4.apache.org,38737,1693807224054/jenkins-hbase4.apache.org%2C38737%2C1693807224054.1693807224237 2023-09-04 06:00:24,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35561,DS-1f884a02-e206-4857-ac2b-56c1647f3b10,DISK], DatanodeInfoWithStorage[127.0.0.1:46443,DS-015b0936-ef74-470f-9c98-0f7973192b02,DISK]] 2023-09-04 06:00:24,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-09-04 06:00:24,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:24,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:00:24,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:00:24,278 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:00:24,281 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-09-04 06:00:24,281 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-09-04 06:00:24,282 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:24,284 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:00:24,285 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:00:24,290 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:00:24,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 06:00:24,297 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=736618, jitterRate=-0.06334207952022552}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 06:00:24,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 06:00:24,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-09-04 06:00:24,300 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-09-04 06:00:24,300 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-09-04 06:00:24,300 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-09-04 06:00:24,301 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-09-04 06:00:24,301 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 0 msec 2023-09-04 06:00:24,301 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-09-04 06:00:24,307 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-09-04 06:00:24,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-09-04 06:00:24,324 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-09-04 06:00:24,324 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-09-04 06:00:24,325 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-09-04 06:00:24,325 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-09-04 06:00:24,326 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-09-04 06:00:24,329 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:24,331 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-09-04 06:00:24,332 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-09-04 06:00:24,333 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-09-04 06:00:24,334 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 06:00:24,334 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 06:00:24,334 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:24,336 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,38737,1693807224054, sessionid=0x10075865c870000, setting cluster-up flag (Was=false) 2023-09-04 06:00:24,339 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:24,345 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-09-04 06:00:24,346 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38737,1693807224054 2023-09-04 06:00:24,351 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:24,356 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-09-04 06:00:24,358 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38737,1693807224054 2023-09-04 06:00:24,359 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/.hbase-snapshot/.tmp 2023-09-04 06:00:24,363 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-09-04 06:00:24,364 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:00:24,364 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:00:24,364 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:00:24,364 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:00:24,364 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-09-04 06:00:24,364 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:24,364 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 06:00:24,364 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:24,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693807254376 2023-09-04 06:00:24,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-09-04 06:00:24,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-09-04 06:00:24,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-09-04 06:00:24,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-09-04 06:00:24,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-09-04 06:00:24,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-09-04 06:00:24,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:24,378 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-09-04 06:00:24,378 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-09-04 06:00:24,378 DEBUG [PEWorker-2] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 06:00:24,378 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-09-04 06:00:24,378 INFO [PEWorker-2] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-09-04 06:00:24,380 INFO [PEWorker-2] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 06:00:24,381 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-09-04 06:00:24,381 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-09-04 06:00:24,381 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693807224381,5,FailOnTimeoutGroup] 2023-09-04 06:00:24,381 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693807224381,5,FailOnTimeoutGroup] 2023-09-04 06:00:24,381 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:24,381 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-09-04 06:00:24,382 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:24,382 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:24,412 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 06:00:24,414 INFO [PEWorker-2] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 06:00:24,414 INFO [PEWorker-2] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3 2023-09-04 06:00:24,434 DEBUG [PEWorker-2] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:24,436 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 06:00:24,439 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/info 2023-09-04 06:00:24,440 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 06:00:24,440 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:24,441 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 06:00:24,442 INFO [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(951): ClusterId : 86709527-0204-4306-9ec8-06aa3e217791 2023-09-04 06:00:24,444 DEBUG [RS:0;jenkins-hbase4:37843] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-09-04 06:00:24,444 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/rep_barrier 2023-09-04 06:00:24,445 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 06:00:24,446 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:24,446 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 06:00:24,448 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/table 2023-09-04 06:00:24,448 DEBUG [RS:0;jenkins-hbase4:37843] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-09-04 06:00:24,448 DEBUG [RS:0;jenkins-hbase4:37843] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-09-04 06:00:24,449 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 06:00:24,450 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:24,450 DEBUG [PEWorker-2] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740 2023-09-04 06:00:24,451 DEBUG [PEWorker-2] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740 2023-09-04 06:00:24,452 DEBUG [RS:0;jenkins-hbase4:37843] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-09-04 06:00:24,453 DEBUG [RS:0;jenkins-hbase4:37843] zookeeper.ReadOnlyZKClient(139): Connect 0x6a10a7d1 to 127.0.0.1:49156 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 06:00:24,455 DEBUG [PEWorker-2] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 06:00:24,457 DEBUG [PEWorker-2] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 06:00:24,460 DEBUG [PEWorker-2] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 06:00:24,461 DEBUG [RS:0;jenkins-hbase4:37843] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@64d754e3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 06:00:24,461 DEBUG [RS:0;jenkins-hbase4:37843] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@60e8cca1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 06:00:24,461 INFO [PEWorker-2] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=713152, jitterRate=-0.0931815356016159}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 06:00:24,462 DEBUG [PEWorker-2] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 06:00:24,462 DEBUG [PEWorker-2] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 06:00:24,462 INFO [PEWorker-2] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 06:00:24,462 DEBUG [PEWorker-2] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 06:00:24,462 DEBUG [PEWorker-2] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 06:00:24,462 DEBUG [PEWorker-2] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 06:00:24,463 INFO [PEWorker-2] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 06:00:24,463 DEBUG [PEWorker-2] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 06:00:24,464 DEBUG [PEWorker-2] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 06:00:24,464 INFO [PEWorker-2] procedure.InitMetaProcedure(103): Going to assign meta 2023-09-04 06:00:24,465 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-09-04 06:00:24,467 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-09-04 06:00:24,469 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-09-04 06:00:24,472 DEBUG [RS:0;jenkins-hbase4:37843] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:37843 2023-09-04 06:00:24,473 INFO [RS:0;jenkins-hbase4:37843] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-09-04 06:00:24,473 INFO [RS:0;jenkins-hbase4:37843] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-09-04 06:00:24,473 DEBUG [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(1022): About to register with Master. 2023-09-04 06:00:24,473 INFO [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38737,1693807224054 with isa=jenkins-hbase4.apache.org/172.31.14.131:37843, startcode=1693807224102 2023-09-04 06:00:24,474 DEBUG [RS:0;jenkins-hbase4:37843] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-09-04 06:00:24,476 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33805, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-09-04 06:00:24,477 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38737] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:24,478 DEBUG [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3 2023-09-04 06:00:24,478 DEBUG [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:43627 2023-09-04 06:00:24,478 DEBUG [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-09-04 06:00:24,481 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 06:00:24,481 DEBUG [RS:0;jenkins-hbase4:37843] zookeeper.ZKUtil(162): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:24,482 WARN [RS:0;jenkins-hbase4:37843] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 06:00:24,482 INFO [RS:0;jenkins-hbase4:37843] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 06:00:24,482 DEBUG [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(1948): logDir=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:24,482 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,37843,1693807224102] 2023-09-04 06:00:24,486 DEBUG [RS:0;jenkins-hbase4:37843] zookeeper.ZKUtil(162): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:24,488 DEBUG [RS:0;jenkins-hbase4:37843] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-09-04 06:00:24,488 INFO [RS:0;jenkins-hbase4:37843] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-09-04 06:00:24,490 INFO [RS:0;jenkins-hbase4:37843] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-09-04 06:00:24,493 INFO [RS:0;jenkins-hbase4:37843] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-09-04 06:00:24,493 INFO [RS:0;jenkins-hbase4:37843] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:24,493 INFO [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-09-04 06:00:24,495 INFO [RS:0;jenkins-hbase4:37843] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:24,495 DEBUG [RS:0;jenkins-hbase4:37843] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:24,495 DEBUG [RS:0;jenkins-hbase4:37843] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:24,495 DEBUG [RS:0;jenkins-hbase4:37843] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:24,495 DEBUG [RS:0;jenkins-hbase4:37843] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:24,495 DEBUG [RS:0;jenkins-hbase4:37843] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:24,495 DEBUG [RS:0;jenkins-hbase4:37843] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 06:00:24,496 DEBUG [RS:0;jenkins-hbase4:37843] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:24,496 DEBUG [RS:0;jenkins-hbase4:37843] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:24,496 DEBUG [RS:0;jenkins-hbase4:37843] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:24,496 DEBUG [RS:0;jenkins-hbase4:37843] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:00:24,497 INFO [RS:0;jenkins-hbase4:37843] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:24,497 INFO [RS:0;jenkins-hbase4:37843] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:24,497 INFO [RS:0;jenkins-hbase4:37843] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:24,511 INFO [RS:0;jenkins-hbase4:37843] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-09-04 06:00:24,511 INFO [RS:0;jenkins-hbase4:37843] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37843,1693807224102-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:24,525 INFO [RS:0;jenkins-hbase4:37843] regionserver.Replication(203): jenkins-hbase4.apache.org,37843,1693807224102 started 2023-09-04 06:00:24,525 INFO [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,37843,1693807224102, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:37843, sessionid=0x10075865c870001 2023-09-04 06:00:24,525 DEBUG [RS:0;jenkins-hbase4:37843] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-09-04 06:00:24,525 DEBUG [RS:0;jenkins-hbase4:37843] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:24,525 DEBUG [RS:0;jenkins-hbase4:37843] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37843,1693807224102' 2023-09-04 06:00:24,525 DEBUG [RS:0;jenkins-hbase4:37843] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 06:00:24,526 DEBUG [RS:0;jenkins-hbase4:37843] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 06:00:24,526 DEBUG [RS:0;jenkins-hbase4:37843] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-09-04 06:00:24,526 DEBUG [RS:0;jenkins-hbase4:37843] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-09-04 06:00:24,526 DEBUG [RS:0;jenkins-hbase4:37843] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:24,526 DEBUG [RS:0;jenkins-hbase4:37843] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37843,1693807224102' 2023-09-04 06:00:24,526 DEBUG [RS:0;jenkins-hbase4:37843] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-09-04 06:00:24,527 DEBUG [RS:0;jenkins-hbase4:37843] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-09-04 06:00:24,527 DEBUG [RS:0;jenkins-hbase4:37843] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-09-04 06:00:24,527 INFO [RS:0;jenkins-hbase4:37843] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-09-04 06:00:24,527 INFO [RS:0;jenkins-hbase4:37843] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-09-04 06:00:24,619 DEBUG [jenkins-hbase4:38737] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-09-04 06:00:24,620 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37843,1693807224102, state=OPENING 2023-09-04 06:00:24,623 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-09-04 06:00:24,624 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:24,625 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37843,1693807224102}] 2023-09-04 06:00:24,625 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 06:00:24,630 INFO [RS:0;jenkins-hbase4:37843] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37843%2C1693807224102, suffix=, logDir=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102, archiveDir=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/oldWALs, maxLogs=32 2023-09-04 06:00:24,648 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35561,DS-1f884a02-e206-4857-ac2b-56c1647f3b10,DISK] 2023-09-04 06:00:24,648 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46443,DS-015b0936-ef74-470f-9c98-0f7973192b02,DISK] 2023-09-04 06:00:24,651 INFO [RS:0;jenkins-hbase4:37843] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102/jenkins-hbase4.apache.org%2C37843%2C1693807224102.1693807224631 2023-09-04 06:00:24,651 DEBUG [RS:0;jenkins-hbase4:37843] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35561,DS-1f884a02-e206-4857-ac2b-56c1647f3b10,DISK], DatanodeInfoWithStorage[127.0.0.1:46443,DS-015b0936-ef74-470f-9c98-0f7973192b02,DISK]] 2023-09-04 06:00:24,780 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:24,780 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-09-04 06:00:24,782 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33322, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-09-04 06:00:24,787 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-09-04 06:00:24,787 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 06:00:24,789 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37843%2C1693807224102.meta, suffix=.meta, logDir=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102, archiveDir=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/oldWALs, maxLogs=32 2023-09-04 06:00:24,807 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46443,DS-015b0936-ef74-470f-9c98-0f7973192b02,DISK] 2023-09-04 06:00:24,808 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35561,DS-1f884a02-e206-4857-ac2b-56c1647f3b10,DISK] 2023-09-04 06:00:24,810 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102/jenkins-hbase4.apache.org%2C37843%2C1693807224102.meta.1693807224792.meta 2023-09-04 06:00:24,811 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46443,DS-015b0936-ef74-470f-9c98-0f7973192b02,DISK], DatanodeInfoWithStorage[127.0.0.1:35561,DS-1f884a02-e206-4857-ac2b-56c1647f3b10,DISK]] 2023-09-04 06:00:24,811 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-09-04 06:00:24,811 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-09-04 06:00:24,811 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-09-04 06:00:24,812 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-09-04 06:00:24,812 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-09-04 06:00:24,812 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:24,812 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-09-04 06:00:24,812 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-09-04 06:00:24,814 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 06:00:24,815 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/info 2023-09-04 06:00:24,815 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/info 2023-09-04 06:00:24,816 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 06:00:24,816 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:24,816 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 06:00:24,818 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/rep_barrier 2023-09-04 06:00:24,818 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/rep_barrier 2023-09-04 06:00:24,818 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 06:00:24,819 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:24,819 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 06:00:24,820 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/table 2023-09-04 06:00:24,820 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/table 2023-09-04 06:00:24,821 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 06:00:24,822 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:24,822 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740 2023-09-04 06:00:24,824 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740 2023-09-04 06:00:24,826 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 06:00:24,828 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 06:00:24,829 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=746067, jitterRate=-0.05132719874382019}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 06:00:24,829 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 06:00:24,831 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693807224779 2023-09-04 06:00:24,836 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-09-04 06:00:24,836 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-09-04 06:00:24,837 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37843,1693807224102, state=OPEN 2023-09-04 06:00:24,839 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-09-04 06:00:24,839 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 06:00:24,842 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-09-04 06:00:24,842 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37843,1693807224102 in 214 msec 2023-09-04 06:00:24,845 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-09-04 06:00:24,846 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 377 msec 2023-09-04 06:00:24,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 485 msec 2023-09-04 06:00:24,848 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693807224848, completionTime=-1 2023-09-04 06:00:24,849 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-09-04 06:00:24,849 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-09-04 06:00:24,852 DEBUG [hconnection-0x49733af4-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 06:00:24,854 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33326, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 06:00:24,856 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-09-04 06:00:24,856 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693807284856 2023-09-04 06:00:24,856 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693807344856 2023-09-04 06:00:24,856 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 7 msec 2023-09-04 06:00:24,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38737,1693807224054-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:24,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38737,1693807224054-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:24,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38737,1693807224054-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:24,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38737, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:24,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-09-04 06:00:24,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-09-04 06:00:24,863 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 06:00:24,864 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-09-04 06:00:24,864 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-09-04 06:00:24,866 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-09-04 06:00:24,867 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-04 06:00:24,869 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/.tmp/data/hbase/namespace/47540753994bd60ea56f493ca9b5ce6f 2023-09-04 06:00:24,869 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/.tmp/data/hbase/namespace/47540753994bd60ea56f493ca9b5ce6f empty. 2023-09-04 06:00:24,870 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/.tmp/data/hbase/namespace/47540753994bd60ea56f493ca9b5ce6f 2023-09-04 06:00:24,870 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-09-04 06:00:24,885 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-09-04 06:00:24,886 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 47540753994bd60ea56f493ca9b5ce6f, NAME => 'hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/.tmp 2023-09-04 06:00:24,900 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:24,900 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 47540753994bd60ea56f493ca9b5ce6f, disabling compactions & flushes 2023-09-04 06:00:24,900 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. 2023-09-04 06:00:24,900 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. 2023-09-04 06:00:24,900 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. after waiting 0 ms 2023-09-04 06:00:24,900 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. 2023-09-04 06:00:24,900 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. 2023-09-04 06:00:24,900 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 47540753994bd60ea56f493ca9b5ce6f: 2023-09-04 06:00:24,905 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-09-04 06:00:24,907 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693807224907"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693807224907"}]},"ts":"1693807224907"} 2023-09-04 06:00:24,910 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-04 06:00:24,911 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-04 06:00:24,911 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693807224911"}]},"ts":"1693807224911"} 2023-09-04 06:00:24,913 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-09-04 06:00:24,922 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=47540753994bd60ea56f493ca9b5ce6f, ASSIGN}] 2023-09-04 06:00:24,924 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=47540753994bd60ea56f493ca9b5ce6f, ASSIGN 2023-09-04 06:00:24,925 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=47540753994bd60ea56f493ca9b5ce6f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37843,1693807224102; forceNewPlan=false, retain=false 2023-09-04 06:00:25,076 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=47540753994bd60ea56f493ca9b5ce6f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:25,077 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693807225076"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693807225076"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693807225076"}]},"ts":"1693807225076"} 2023-09-04 06:00:25,080 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 47540753994bd60ea56f493ca9b5ce6f, server=jenkins-hbase4.apache.org,37843,1693807224102}] 2023-09-04 06:00:25,237 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. 2023-09-04 06:00:25,238 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 47540753994bd60ea56f493ca9b5ce6f, NAME => 'hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f.', STARTKEY => '', ENDKEY => ''} 2023-09-04 06:00:25,238 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 47540753994bd60ea56f493ca9b5ce6f 2023-09-04 06:00:25,238 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:25,238 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 47540753994bd60ea56f493ca9b5ce6f 2023-09-04 06:00:25,238 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 47540753994bd60ea56f493ca9b5ce6f 2023-09-04 06:00:25,240 INFO [StoreOpener-47540753994bd60ea56f493ca9b5ce6f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 47540753994bd60ea56f493ca9b5ce6f 2023-09-04 06:00:25,242 DEBUG [StoreOpener-47540753994bd60ea56f493ca9b5ce6f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/namespace/47540753994bd60ea56f493ca9b5ce6f/info 2023-09-04 06:00:25,242 DEBUG [StoreOpener-47540753994bd60ea56f493ca9b5ce6f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/namespace/47540753994bd60ea56f493ca9b5ce6f/info 2023-09-04 06:00:25,243 INFO [StoreOpener-47540753994bd60ea56f493ca9b5ce6f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 47540753994bd60ea56f493ca9b5ce6f columnFamilyName info 2023-09-04 06:00:25,244 INFO [StoreOpener-47540753994bd60ea56f493ca9b5ce6f-1] regionserver.HStore(310): Store=47540753994bd60ea56f493ca9b5ce6f/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:25,245 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/namespace/47540753994bd60ea56f493ca9b5ce6f 2023-09-04 06:00:25,246 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/namespace/47540753994bd60ea56f493ca9b5ce6f 2023-09-04 06:00:25,250 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 47540753994bd60ea56f493ca9b5ce6f 2023-09-04 06:00:25,252 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/namespace/47540753994bd60ea56f493ca9b5ce6f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 06:00:25,253 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 47540753994bd60ea56f493ca9b5ce6f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=773326, jitterRate=-0.016665205359458923}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 06:00:25,253 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 47540753994bd60ea56f493ca9b5ce6f: 2023-09-04 06:00:25,254 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f., pid=6, masterSystemTime=1693807225232 2023-09-04 06:00:25,259 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. 2023-09-04 06:00:25,259 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. 2023-09-04 06:00:25,260 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=47540753994bd60ea56f493ca9b5ce6f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:25,260 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693807225259"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693807225259"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693807225259"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693807225259"}]},"ts":"1693807225259"} 2023-09-04 06:00:25,267 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-09-04 06:00:25,267 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 47540753994bd60ea56f493ca9b5ce6f, server=jenkins-hbase4.apache.org,37843,1693807224102 in 184 msec 2023-09-04 06:00:25,270 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-09-04 06:00:25,271 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=47540753994bd60ea56f493ca9b5ce6f, ASSIGN in 345 msec 2023-09-04 06:00:25,272 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-04 06:00:25,272 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693807225272"}]},"ts":"1693807225272"} 2023-09-04 06:00:25,274 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-09-04 06:00:25,278 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-09-04 06:00:25,280 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 415 msec 2023-09-04 06:00:25,366 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-09-04 06:00:25,367 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-09-04 06:00:25,367 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:25,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-09-04 06:00:25,383 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 06:00:25,387 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 14 msec 2023-09-04 06:00:25,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-09-04 06:00:25,403 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 06:00:25,407 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 12 msec 2023-09-04 06:00:25,420 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-09-04 06:00:25,423 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-09-04 06:00:25,423 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.274sec 2023-09-04 06:00:25,423 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-09-04 06:00:25,423 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-09-04 06:00:25,423 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-09-04 06:00:25,423 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38737,1693807224054-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-09-04 06:00:25,423 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38737,1693807224054-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-09-04 06:00:25,425 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-09-04 06:00:25,443 DEBUG [Listener at localhost/35025] zookeeper.ReadOnlyZKClient(139): Connect 0x6cf7f757 to 127.0.0.1:49156 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 06:00:25,448 DEBUG [Listener at localhost/35025] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@260d2cd1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 06:00:25,450 DEBUG [hconnection-0x281bd895-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 06:00:25,452 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33664, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 06:00:25,454 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,38737,1693807224054 2023-09-04 06:00:25,454 INFO [Listener at localhost/35025] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:00:25,458 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-09-04 06:00:25,458 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:00:25,459 INFO [Listener at localhost/35025] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-09-04 06:00:25,461 DEBUG [Listener at localhost/35025] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-09-04 06:00:25,463 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60076, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-09-04 06:00:25,465 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-09-04 06:00:25,465 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-09-04 06:00:25,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-04 06:00:25,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] procedure2.ProcedureExecutor(1028): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:25,469 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-09-04 06:00:25,469 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testCompactionRecordDoesntBlockRolling" procId is: 9 2023-09-04 06:00:25,470 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-04 06:00:25,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-04 06:00:25,473 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6 2023-09-04 06:00:25,474 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6 empty. 2023-09-04 06:00:25,475 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6 2023-09-04 06:00:25,475 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testCompactionRecordDoesntBlockRolling regions 2023-09-04 06:00:25,504 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/.tabledesc/.tableinfo.0000000001 2023-09-04 06:00:25,505 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => 96ba21e9bc791d113f0f7f6ea8c0e0a6, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/.tmp 2023-09-04 06:00:25,526 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:25,526 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1604): Closing 96ba21e9bc791d113f0f7f6ea8c0e0a6, disabling compactions & flushes 2023-09-04 06:00:25,526 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:00:25,526 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:00:25,526 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. after waiting 0 ms 2023-09-04 06:00:25,526 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:00:25,526 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:00:25,526 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1558): Region close journal for 96ba21e9bc791d113f0f7f6ea8c0e0a6: 2023-09-04 06:00:25,530 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ADD_TO_META 2023-09-04 06:00:25,532 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1693807225531"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693807225531"}]},"ts":"1693807225531"} 2023-09-04 06:00:25,534 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-04 06:00:25,536 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-04 06:00:25,536 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693807225536"}]},"ts":"1693807225536"} 2023-09-04 06:00:25,538 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLING in hbase:meta 2023-09-04 06:00:25,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=96ba21e9bc791d113f0f7f6ea8c0e0a6, ASSIGN}] 2023-09-04 06:00:25,545 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=96ba21e9bc791d113f0f7f6ea8c0e0a6, ASSIGN 2023-09-04 06:00:25,547 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=96ba21e9bc791d113f0f7f6ea8c0e0a6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37843,1693807224102; forceNewPlan=false, retain=false 2023-09-04 06:00:25,698 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=96ba21e9bc791d113f0f7f6ea8c0e0a6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:25,698 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1693807225698"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693807225698"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693807225698"}]},"ts":"1693807225698"} 2023-09-04 06:00:25,701 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 96ba21e9bc791d113f0f7f6ea8c0e0a6, server=jenkins-hbase4.apache.org,37843,1693807224102}] 2023-09-04 06:00:25,709 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-04 06:00:25,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:00:25,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 96ba21e9bc791d113f0f7f6ea8c0e0a6, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6.', STARTKEY => '', ENDKEY => ''} 2023-09-04 06:00:25,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testCompactionRecordDoesntBlockRolling 96ba21e9bc791d113f0f7f6ea8c0e0a6 2023-09-04 06:00:25,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:00:25,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 96ba21e9bc791d113f0f7f6ea8c0e0a6 2023-09-04 06:00:25,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 96ba21e9bc791d113f0f7f6ea8c0e0a6 2023-09-04 06:00:25,862 INFO [StoreOpener-96ba21e9bc791d113f0f7f6ea8c0e0a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 96ba21e9bc791d113f0f7f6ea8c0e0a6 2023-09-04 06:00:25,863 DEBUG [StoreOpener-96ba21e9bc791d113f0f7f6ea8c0e0a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info 2023-09-04 06:00:25,864 DEBUG [StoreOpener-96ba21e9bc791d113f0f7f6ea8c0e0a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info 2023-09-04 06:00:25,864 INFO [StoreOpener-96ba21e9bc791d113f0f7f6ea8c0e0a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 96ba21e9bc791d113f0f7f6ea8c0e0a6 columnFamilyName info 2023-09-04 06:00:25,865 INFO [StoreOpener-96ba21e9bc791d113f0f7f6ea8c0e0a6-1] regionserver.HStore(310): Store=96ba21e9bc791d113f0f7f6ea8c0e0a6/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:00:25,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6 2023-09-04 06:00:25,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6 2023-09-04 06:00:25,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 96ba21e9bc791d113f0f7f6ea8c0e0a6 2023-09-04 06:00:25,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 06:00:25,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 96ba21e9bc791d113f0f7f6ea8c0e0a6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=834029, jitterRate=0.060522958636283875}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 06:00:25,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 96ba21e9bc791d113f0f7f6ea8c0e0a6: 2023-09-04 06:00:25,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6., pid=11, masterSystemTime=1693807225855 2023-09-04 06:00:25,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:00:25,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:00:25,877 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=96ba21e9bc791d113f0f7f6ea8c0e0a6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:25,877 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1693807225877"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693807225877"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693807225877"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693807225877"}]},"ts":"1693807225877"} 2023-09-04 06:00:25,882 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=11, resume processing ppid=10 2023-09-04 06:00:25,882 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 96ba21e9bc791d113f0f7f6ea8c0e0a6, server=jenkins-hbase4.apache.org,37843,1693807224102 in 179 msec 2023-09-04 06:00:25,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=10, resume processing ppid=9 2023-09-04 06:00:25,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=96ba21e9bc791d113f0f7f6ea8c0e0a6, ASSIGN in 339 msec 2023-09-04 06:00:25,886 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-04 06:00:25,887 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693807225887"}]},"ts":"1693807225887"} 2023-09-04 06:00:25,889 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLED in hbase:meta 2023-09-04 06:00:25,891 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_POST_OPERATION 2023-09-04 06:00:25,893 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling in 425 msec 2023-09-04 06:00:29,637 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-04 06:00:29,637 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-09-04 06:00:30,742 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-04 06:00:30,783 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-09-04 06:00:30,783 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-09-04 06:00:30,784 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 06:00:35,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-04 06:00:35,475 INFO [Listener at localhost/35025] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testCompactionRecordDoesntBlockRolling, procId: 9 completed 2023-09-04 06:00:35,478 DEBUG [Listener at localhost/35025] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:35,478 DEBUG [Listener at localhost/35025] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:00:35,491 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-09-04 06:00:35,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] procedure.ProcedureCoordinator(165): Submitting procedure hbase:namespace 2023-09-04 06:00:35,498 INFO [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'hbase:namespace' 2023-09-04 06:00:35,499 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 06:00:35,499 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'hbase:namespace' starting 'acquire' 2023-09-04 06:00:35,499 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'hbase:namespace', kicking off acquire phase on members. 2023-09-04 06:00:35,500 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 06:00:35,500 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-09-04 06:00:35,501 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 06:00:35,501 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 06:00:35,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 06:00:35,502 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,502 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-09-04 06:00:35,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/hbase:namespace 2023-09-04 06:00:35,502 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 06:00:35,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-09-04 06:00:35,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-09-04 06:00:35,503 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for hbase:namespace 2023-09-04 06:00:35,505 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:hbase:namespace 2023-09-04 06:00:35,505 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'hbase:namespace' with timeout 60000ms 2023-09-04 06:00:35,505 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 06:00:35,506 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'hbase:namespace' starting 'acquire' stage 2023-09-04 06:00:35,507 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-09-04 06:00:35,507 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. 2023-09-04 06:00:35,507 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-09-04 06:00:35,507 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. started... 2023-09-04 06:00:35,507 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 47540753994bd60ea56f493ca9b5ce6f 1/1 column families, dataSize=78 B heapSize=488 B 2023-09-04 06:00:35,522 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/namespace/47540753994bd60ea56f493ca9b5ce6f/.tmp/info/7cc91c71f1bf4793a330f7e6b064e47b 2023-09-04 06:00:35,532 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/namespace/47540753994bd60ea56f493ca9b5ce6f/.tmp/info/7cc91c71f1bf4793a330f7e6b064e47b as hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/namespace/47540753994bd60ea56f493ca9b5ce6f/info/7cc91c71f1bf4793a330f7e6b064e47b 2023-09-04 06:00:35,540 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/namespace/47540753994bd60ea56f493ca9b5ce6f/info/7cc91c71f1bf4793a330f7e6b064e47b, entries=2, sequenceid=6, filesize=4.8 K 2023-09-04 06:00:35,542 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 47540753994bd60ea56f493ca9b5ce6f in 35ms, sequenceid=6, compaction requested=false 2023-09-04 06:00:35,543 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 47540753994bd60ea56f493ca9b5ce6f: 2023-09-04 06:00:35,543 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. 2023-09-04 06:00:35,543 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-09-04 06:00:35,543 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-09-04 06:00:35,543 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,543 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'hbase:namespace' locally acquired 2023-09-04 06:00:35,543 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37843,1693807224102' joining acquired barrier for procedure (hbase:namespace) in zk 2023-09-04 06:00:35,545 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,545 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 06:00:35,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 06:00:35,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 06:00:35,546 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 06:00:35,546 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'hbase:namespace' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-09-04 06:00:35,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 06:00:35,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 06:00:35,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-04 06:00:35,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 06:00:35,548 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37843,1693807224102' joining acquired barrier for procedure 'hbase:namespace' on coordinator 2023-09-04 06:00:35,548 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74352653[Count = 0] remaining members to acquire global barrier 2023-09-04 06:00:35,548 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'hbase:namespace' starting 'in-barrier' execution. 2023-09-04 06:00:35,548 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 06:00:35,550 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 06:00:35,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 06:00:35,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 06:00:35,551 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'hbase:namespace' received 'reached' from coordinator. 2023-09-04 06:00:35,551 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'hbase:namespace' locally completed 2023-09-04 06:00:35,551 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'hbase:namespace' completed for member 'jenkins-hbase4.apache.org,37843,1693807224102' in zk 2023-09-04 06:00:35,551 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,551 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-09-04 06:00:35,553 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'hbase:namespace' has notified controller of completion 2023-09-04 06:00:35,553 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,553 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 06:00:35,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 06:00:35,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 06:00:35,553 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'hbase:namespace' completed. 2023-09-04 06:00:35,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 06:00:35,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 06:00:35,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-04 06:00:35,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 06:00:35,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-04 06:00:35,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'hbase:namespace' member 'jenkins-hbase4.apache.org,37843,1693807224102': 2023-09-04 06:00:35,557 INFO [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'hbase:namespace' execution completed 2023-09-04 06:00:35,557 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-09-04 06:00:35,557 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37843,1693807224102' released barrier for procedure'hbase:namespace', counting down latch. Waiting for 0 more 2023-09-04 06:00:35,557 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-09-04 06:00:35,557 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:hbase:namespace 2023-09-04 06:00:35,557 INFO [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure hbase:namespaceincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-09-04 06:00:35,559 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 06:00:35,559 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 06:00:35,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 06:00:35,559 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 06:00:35,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 06:00:35,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 06:00:35,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 06:00:35,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 06:00:35,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 06:00:35,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 06:00:35,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 06:00:35,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 06:00:35,560 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-04 06:00:35,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 06:00:35,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-04 06:00:35,562 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 06:00:35,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-04 06:00:35,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,568 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,568 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-09-04 06:00:35,568 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 06:00:35,568 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-09-04 06:00:35,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 06:00:35,568 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 06:00:35,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 06:00:35,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'hbase:namespace' 2023-09-04 06:00:35,568 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:35,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-09-04 06:00:35,568 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 06:00:35,569 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 06:00:35,570 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 06:00:35,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 06:00:35,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 06:00:35,570 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 06:00:35,571 DEBUG [Listener at localhost/35025] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : hbase:namespace'' to complete. (max 20000 ms per retry) 2023-09-04 06:00:35,571 DEBUG [Listener at localhost/35025] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-09-04 06:00:45,571 DEBUG [Listener at localhost/35025] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-09-04 06:00:45,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-09-04 06:00:45,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-09-04 06:00:45,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,592 INFO [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 06:00:45,592 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 06:00:45,593 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-09-04 06:00:45,593 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-09-04 06:00:45,593 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,593 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,596 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 06:00:45,596 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 06:00:45,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 06:00:45,596 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,596 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-09-04 06:00:45,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-09-04 06:00:45,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,597 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,598 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,598 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-09-04 06:00:45,598 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 06:00:45,598 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-09-04 06:00:45,599 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-09-04 06:00:45,599 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-09-04 06:00:45,599 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:00:45,599 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. started... 2023-09-04 06:00:45,599 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 96ba21e9bc791d113f0f7f6ea8c0e0a6 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-04 06:00:45,621 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=5 (bloomFilter=true), to=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/.tmp/info/063538088e764474947ba3a3b145c48a 2023-09-04 06:00:45,633 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/.tmp/info/063538088e764474947ba3a3b145c48a as hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/063538088e764474947ba3a3b145c48a 2023-09-04 06:00:45,641 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/063538088e764474947ba3a3b145c48a, entries=1, sequenceid=5, filesize=5.8 K 2023-09-04 06:00:45,643 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 96ba21e9bc791d113f0f7f6ea8c0e0a6 in 43ms, sequenceid=5, compaction requested=false 2023-09-04 06:00:45,643 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 96ba21e9bc791d113f0f7f6ea8c0e0a6: 2023-09-04 06:00:45,643 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:00:45,643 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-09-04 06:00:45,644 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-09-04 06:00:45,644 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,644 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-09-04 06:00:45,644 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37843,1693807224102' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-09-04 06:00:45,645 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,645 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 06:00:45,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 06:00:45,646 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,646 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-09-04 06:00:45,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 06:00:45,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 06:00:45,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 06:00:45,648 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37843,1693807224102' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-09-04 06:00:45,648 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@700abc0d[Count = 0] remaining members to acquire global barrier 2023-09-04 06:00:45,648 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-09-04 06:00:45,648 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,649 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,649 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-09-04 06:00:45,649 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-09-04 06:00:45,649 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,649 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-09-04 06:00:45,649 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,37843,1693807224102' in zk 2023-09-04 06:00:45,651 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-09-04 06:00:45,651 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,651 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 06:00:45,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,651 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-09-04 06:00:45,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 06:00:45,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 06:00:45,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 06:00:45,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 06:00:45,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 06:00:45,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,37843,1693807224102': 2023-09-04 06:00:45,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37843,1693807224102' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-09-04 06:00:45,655 INFO [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-09-04 06:00:45,655 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-09-04 06:00:45,655 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-09-04 06:00:45,655 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,655 INFO [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-09-04 06:00:45,660 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,660 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 06:00:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 06:00:45,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,660 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 06:00:45,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 06:00:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 06:00:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 06:00:45,661 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 06:00:45,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,668 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 06:00:45,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,671 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 06:00:45,671 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 06:00:45,671 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,671 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 06:00:45,671 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 06:00:45,672 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:45,671 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-09-04 06:00:45,671 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 06:00:45,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-09-04 06:00:45,671 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 06:00:45,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 06:00:45,672 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 06:00:45,673 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,673 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:45,673 DEBUG [Listener at localhost/35025] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-09-04 06:00:45,673 DEBUG [Listener at localhost/35025] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-09-04 06:00:55,673 DEBUG [Listener at localhost/35025] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-09-04 06:00:55,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-09-04 06:00:55,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-09-04 06:00:55,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-09-04 06:00:55,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,684 INFO [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 06:00:55,684 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 06:00:55,685 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-09-04 06:00:55,685 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-09-04 06:00:55,685 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,685 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,688 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,688 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 06:00:55,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 06:00:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 06:00:55,688 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,688 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-09-04 06:00:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-09-04 06:00:55,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,689 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,689 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-09-04 06:00:55,689 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,690 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-09-04 06:00:55,690 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 06:00:55,690 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-09-04 06:00:55,690 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-09-04 06:00:55,690 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-09-04 06:00:55,690 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:00:55,690 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. started... 2023-09-04 06:00:55,691 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 96ba21e9bc791d113f0f7f6ea8c0e0a6 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-04 06:00:55,704 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=9 (bloomFilter=true), to=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/.tmp/info/a29b3883ef4d4b08809b39a63b8047ff 2023-09-04 06:00:55,714 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/.tmp/info/a29b3883ef4d4b08809b39a63b8047ff as hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/a29b3883ef4d4b08809b39a63b8047ff 2023-09-04 06:00:55,721 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/a29b3883ef4d4b08809b39a63b8047ff, entries=1, sequenceid=9, filesize=5.8 K 2023-09-04 06:00:55,723 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 96ba21e9bc791d113f0f7f6ea8c0e0a6 in 32ms, sequenceid=9, compaction requested=false 2023-09-04 06:00:55,723 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 96ba21e9bc791d113f0f7f6ea8c0e0a6: 2023-09-04 06:00:55,723 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:00:55,723 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-09-04 06:00:55,723 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-09-04 06:00:55,723 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,724 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-09-04 06:00:55,724 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37843,1693807224102' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-09-04 06:00:55,725 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,725 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 06:00:55,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 06:00:55,726 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,726 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-09-04 06:00:55,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 06:00:55,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 06:00:55,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 06:00:55,728 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37843,1693807224102' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-09-04 06:00:55,728 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35082c68[Count = 0] remaining members to acquire global barrier 2023-09-04 06:00:55,728 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-09-04 06:00:55,729 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,730 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,730 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-09-04 06:00:55,730 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-09-04 06:00:55,730 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,730 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-09-04 06:00:55,730 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,37843,1693807224102' in zk 2023-09-04 06:00:55,732 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-09-04 06:00:55,732 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,732 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 06:00:55,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 06:00:55,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 06:00:55,732 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-09-04 06:00:55,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 06:00:55,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 06:00:55,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 06:00:55,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,37843,1693807224102': 2023-09-04 06:00:55,736 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37843,1693807224102' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-09-04 06:00:55,736 INFO [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-09-04 06:00:55,736 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-09-04 06:00:55,736 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-09-04 06:00:55,736 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,736 INFO [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-09-04 06:00:55,738 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,738 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 06:00:55,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,738 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 06:00:55,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 06:00:55,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 06:00:55,739 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 06:00:55,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 06:00:55,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 06:00:55,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,741 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 06:00:55,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,743 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,743 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 06:00:55,743 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 06:00:55,743 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 06:00:55,743 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-09-04 06:00:55,743 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 06:00:55,743 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,743 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 06:00:55,744 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:00:55,744 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-09-04 06:00:55,744 DEBUG [Listener at localhost/35025] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-09-04 06:00:55,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 06:00:55,744 DEBUG [Listener at localhost/35025] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-09-04 06:00:55,744 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,745 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,745 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:00:55,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 06:00:55,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 06:01:05,745 DEBUG [Listener at localhost/35025] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-09-04 06:01:05,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-09-04 06:01:05,765 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46443,DS-015b0936-ef74-470f-9c98-0f7973192b02,DISK] 2023-09-04 06:01:05,766 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35561,DS-1f884a02-e206-4857-ac2b-56c1647f3b10,DISK] 2023-09-04 06:01:05,769 INFO [Listener at localhost/35025] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102/jenkins-hbase4.apache.org%2C37843%2C1693807224102.1693807224631 with entries=13, filesize=6.36 KB; new WAL /user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102/jenkins-hbase4.apache.org%2C37843%2C1693807224102.1693807265749 2023-09-04 06:01:05,770 DEBUG [Listener at localhost/35025] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46443,DS-015b0936-ef74-470f-9c98-0f7973192b02,DISK], DatanodeInfoWithStorage[127.0.0.1:35561,DS-1f884a02-e206-4857-ac2b-56c1647f3b10,DISK]] 2023-09-04 06:01:05,770 DEBUG [Listener at localhost/35025] wal.AbstractFSWAL(716): hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102/jenkins-hbase4.apache.org%2C37843%2C1693807224102.1693807224631 is not closed yet, will try archiving it next time 2023-09-04 06:01:05,775 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-09-04 06:01:05,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-09-04 06:01:05,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,778 INFO [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 06:01:05,778 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 06:01:05,778 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-09-04 06:01:05,778 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-09-04 06:01:05,779 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,779 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,781 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 06:01:05,781 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 06:01:05,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 06:01:05,781 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,782 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-09-04 06:01:05,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,782 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-09-04 06:01:05,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,783 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-09-04 06:01:05,783 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,783 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-09-04 06:01:05,783 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 06:01:05,783 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-09-04 06:01:05,783 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-09-04 06:01:05,783 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-09-04 06:01:05,784 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:01:05,784 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. started... 2023-09-04 06:01:05,784 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 96ba21e9bc791d113f0f7f6ea8c0e0a6 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-04 06:01:05,800 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=13 (bloomFilter=true), to=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/.tmp/info/4d7935d37f184b179a1b46c2388b00aa 2023-09-04 06:01:05,809 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/.tmp/info/4d7935d37f184b179a1b46c2388b00aa as hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/4d7935d37f184b179a1b46c2388b00aa 2023-09-04 06:01:05,816 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/4d7935d37f184b179a1b46c2388b00aa, entries=1, sequenceid=13, filesize=5.8 K 2023-09-04 06:01:05,817 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 96ba21e9bc791d113f0f7f6ea8c0e0a6 in 33ms, sequenceid=13, compaction requested=true 2023-09-04 06:01:05,818 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 96ba21e9bc791d113f0f7f6ea8c0e0a6: 2023-09-04 06:01:05,818 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:01:05,818 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-09-04 06:01:05,818 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-09-04 06:01:05,818 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,818 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-09-04 06:01:05,818 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37843,1693807224102' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-09-04 06:01:05,820 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,820 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 06:01:05,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 06:01:05,820 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,821 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-09-04 06:01:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 06:01:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 06:01:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 06:01:05,822 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37843,1693807224102' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-09-04 06:01:05,822 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59ad4c8b[Count = 0] remaining members to acquire global barrier 2023-09-04 06:01:05,822 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-09-04 06:01:05,822 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,823 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,824 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-09-04 06:01:05,824 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-09-04 06:01:05,824 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,824 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-09-04 06:01:05,824 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,37843,1693807224102' in zk 2023-09-04 06:01:05,825 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,825 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-09-04 06:01:05,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 06:01:05,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 06:01:05,825 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 06:01:05,826 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-09-04 06:01:05,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 06:01:05,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 06:01:05,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 06:01:05,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,37843,1693807224102': 2023-09-04 06:01:05,828 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37843,1693807224102' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-09-04 06:01:05,828 INFO [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-09-04 06:01:05,829 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-09-04 06:01:05,829 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-09-04 06:01:05,829 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,829 INFO [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-09-04 06:01:05,831 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,831 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 06:01:05,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 06:01:05,831 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 06:01:05,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 06:01:05,832 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 06:01:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 06:01:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 06:01:05,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,833 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 06:01:05,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,836 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,836 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,836 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 06:01:05,836 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 06:01:05,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 06:01:05,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 06:01:05,836 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,836 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 06:01:05,837 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:05,837 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,836 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-09-04 06:01:05,836 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 06:01:05,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 06:01:05,837 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-09-04 06:01:05,837 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 06:01:05,837 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:05,838 DEBUG [Listener at localhost/35025] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-09-04 06:01:05,838 DEBUG [Listener at localhost/35025] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-09-04 06:01:15,838 DEBUG [Listener at localhost/35025] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-09-04 06:01:15,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-09-04 06:01:15,840 DEBUG [Listener at localhost/35025] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 06:01:15,846 DEBUG [Listener at localhost/35025] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 17694 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 06:01:15,847 DEBUG [Listener at localhost/35025] regionserver.HStore(1912): 96ba21e9bc791d113f0f7f6ea8c0e0a6/info is initiating minor compaction (all files) 2023-09-04 06:01:15,848 INFO [Listener at localhost/35025] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-09-04 06:01:15,848 INFO [Listener at localhost/35025] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:15,848 INFO [Listener at localhost/35025] regionserver.HRegion(2259): Starting compaction of 96ba21e9bc791d113f0f7f6ea8c0e0a6/info in TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:01:15,848 INFO [Listener at localhost/35025] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/063538088e764474947ba3a3b145c48a, hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/a29b3883ef4d4b08809b39a63b8047ff, hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/4d7935d37f184b179a1b46c2388b00aa] into tmpdir=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/.tmp, totalSize=17.3 K 2023-09-04 06:01:15,849 DEBUG [Listener at localhost/35025] compactions.Compactor(207): Compacting 063538088e764474947ba3a3b145c48a, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1693807245583 2023-09-04 06:01:15,850 DEBUG [Listener at localhost/35025] compactions.Compactor(207): Compacting a29b3883ef4d4b08809b39a63b8047ff, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1693807255676 2023-09-04 06:01:15,850 DEBUG [Listener at localhost/35025] compactions.Compactor(207): Compacting 4d7935d37f184b179a1b46c2388b00aa, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1693807265747 2023-09-04 06:01:15,872 INFO [Listener at localhost/35025] throttle.PressureAwareThroughputController(145): 96ba21e9bc791d113f0f7f6ea8c0e0a6#info#compaction#9 average throughput is 3.01 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 06:01:15,949 DEBUG [Listener at localhost/35025] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/.tmp/info/b69f1014c76c4321a8c2d943962f4846 as hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/b69f1014c76c4321a8c2d943962f4846 2023-09-04 06:01:15,968 INFO [Listener at localhost/35025] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 96ba21e9bc791d113f0f7f6ea8c0e0a6/info of 96ba21e9bc791d113f0f7f6ea8c0e0a6 into b69f1014c76c4321a8c2d943962f4846(size=7.9 K), total size for store is 7.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 06:01:15,968 DEBUG [Listener at localhost/35025] regionserver.HRegion(2289): Compaction status journal for 96ba21e9bc791d113f0f7f6ea8c0e0a6: 2023-09-04 06:01:15,986 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46443,DS-015b0936-ef74-470f-9c98-0f7973192b02,DISK] 2023-09-04 06:01:15,986 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35561,DS-1f884a02-e206-4857-ac2b-56c1647f3b10,DISK] 2023-09-04 06:01:15,989 INFO [Listener at localhost/35025] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102/jenkins-hbase4.apache.org%2C37843%2C1693807224102.1693807265749 with entries=4, filesize=2.43 KB; new WAL /user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102/jenkins-hbase4.apache.org%2C37843%2C1693807224102.1693807275970 2023-09-04 06:01:15,989 DEBUG [Listener at localhost/35025] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46443,DS-015b0936-ef74-470f-9c98-0f7973192b02,DISK], DatanodeInfoWithStorage[127.0.0.1:35561,DS-1f884a02-e206-4857-ac2b-56c1647f3b10,DISK]] 2023-09-04 06:01:15,989 DEBUG [Listener at localhost/35025] wal.AbstractFSWAL(716): hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102/jenkins-hbase4.apache.org%2C37843%2C1693807224102.1693807265749 is not closed yet, will try archiving it next time 2023-09-04 06:01:15,990 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102/jenkins-hbase4.apache.org%2C37843%2C1693807224102.1693807224631 to hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/oldWALs/jenkins-hbase4.apache.org%2C37843%2C1693807224102.1693807224631 2023-09-04 06:01:15,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-09-04 06:01:15,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-09-04 06:01:15,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:15,998 INFO [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 06:01:15,998 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 06:01:15,998 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-09-04 06:01:15,998 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-09-04 06:01:15,999 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:15,999 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,004 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,004 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 06:01:16,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 06:01:16,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 06:01:16,004 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,004 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-09-04 06:01:16,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,005 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-09-04 06:01:16,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,005 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,005 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,006 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-09-04 06:01:16,006 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 06:01:16,006 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-09-04 06:01:16,006 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-09-04 06:01:16,006 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-09-04 06:01:16,006 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:01:16,006 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. started... 2023-09-04 06:01:16,007 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 96ba21e9bc791d113f0f7f6ea8c0e0a6 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-04 06:01:16,024 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=18 (bloomFilter=true), to=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/.tmp/info/726b83cf7ddb45bf81246d0cc81244b2 2023-09-04 06:01:16,033 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/.tmp/info/726b83cf7ddb45bf81246d0cc81244b2 as hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/726b83cf7ddb45bf81246d0cc81244b2 2023-09-04 06:01:16,040 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/726b83cf7ddb45bf81246d0cc81244b2, entries=1, sequenceid=18, filesize=5.8 K 2023-09-04 06:01:16,041 INFO [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 96ba21e9bc791d113f0f7f6ea8c0e0a6 in 34ms, sequenceid=18, compaction requested=false 2023-09-04 06:01:16,041 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 96ba21e9bc791d113f0f7f6ea8c0e0a6: 2023-09-04 06:01:16,041 DEBUG [rs(jenkins-hbase4.apache.org,37843,1693807224102)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:01:16,041 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-09-04 06:01:16,042 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-09-04 06:01:16,042 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,042 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-09-04 06:01:16,042 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37843,1693807224102' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-09-04 06:01:16,044 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,044 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 06:01:16,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 06:01:16,044 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,044 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-09-04 06:01:16,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 06:01:16,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 06:01:16,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 06:01:16,046 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37843,1693807224102' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-09-04 06:01:16,046 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f7fe53[Count = 0] remaining members to acquire global barrier 2023-09-04 06:01:16,046 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-09-04 06:01:16,046 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,048 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,048 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-09-04 06:01:16,048 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,048 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-09-04 06:01:16,048 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-09-04 06:01:16,049 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,37843,1693807224102' in zk 2023-09-04 06:01:16,050 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-09-04 06:01:16,051 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 06:01:16,050 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,051 DEBUG [member: 'jenkins-hbase4.apache.org,37843,1693807224102' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-09-04 06:01:16,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 06:01:16,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 06:01:16,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 06:01:16,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 06:01:16,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 06:01:16,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,37843,1693807224102': 2023-09-04 06:01:16,054 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37843,1693807224102' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-09-04 06:01:16,054 INFO [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-09-04 06:01:16,054 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-09-04 06:01:16,054 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-09-04 06:01:16,054 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,054 INFO [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-09-04 06:01:16,056 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,056 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 06:01:16,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 06:01:16,056 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 06:01:16,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 06:01:16,056 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 06:01:16,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 06:01:16,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 06:01:16,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,058 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 06:01:16,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,060 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,060 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,060 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,060 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 06:01:16,060 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:16,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 06:01:16,060 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 06:01:16,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-09-04 06:01:16,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 06:01:16,060 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,060 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-09-04 06:01:16,060 DEBUG [(jenkins-hbase4.apache.org,38737,1693807224054)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 06:01:16,060 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 06:01:16,061 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,061 DEBUG [Listener at localhost/35025] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-09-04 06:01:16,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 06:01:16,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 06:01:16,061 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 06:01:16,062 DEBUG [Listener at localhost/35025] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-09-04 06:01:26,062 DEBUG [Listener at localhost/35025] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-09-04 06:01:26,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-09-04 06:01:26,082 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46443,DS-015b0936-ef74-470f-9c98-0f7973192b02,DISK] 2023-09-04 06:01:26,083 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35561,DS-1f884a02-e206-4857-ac2b-56c1647f3b10,DISK] 2023-09-04 06:01:26,085 INFO [Listener at localhost/35025] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102/jenkins-hbase4.apache.org%2C37843%2C1693807224102.1693807275970 with entries=3, filesize=1.94 KB; new WAL /user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102/jenkins-hbase4.apache.org%2C37843%2C1693807224102.1693807286066 2023-09-04 06:01:26,085 DEBUG [Listener at localhost/35025] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46443,DS-015b0936-ef74-470f-9c98-0f7973192b02,DISK], DatanodeInfoWithStorage[127.0.0.1:35561,DS-1f884a02-e206-4857-ac2b-56c1647f3b10,DISK]] 2023-09-04 06:01:26,085 DEBUG [Listener at localhost/35025] wal.AbstractFSWAL(716): hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102/jenkins-hbase4.apache.org%2C37843%2C1693807224102.1693807275970 is not closed yet, will try archiving it next time 2023-09-04 06:01:26,085 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/WALs/jenkins-hbase4.apache.org,37843,1693807224102/jenkins-hbase4.apache.org%2C37843%2C1693807224102.1693807265749 to hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/oldWALs/jenkins-hbase4.apache.org%2C37843%2C1693807224102.1693807265749 2023-09-04 06:01:26,085 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-09-04 06:01:26,086 INFO [Listener at localhost/35025] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-09-04 06:01:26,086 DEBUG [Listener at localhost/35025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6cf7f757 to 127.0.0.1:49156 2023-09-04 06:01:26,086 DEBUG [Listener at localhost/35025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:01:26,086 DEBUG [Listener at localhost/35025] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-09-04 06:01:26,086 DEBUG [Listener at localhost/35025] util.JVMClusterUtil(257): Found active master hash=759474164, stopped=false 2023-09-04 06:01:26,086 INFO [Listener at localhost/35025] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38737,1693807224054 2023-09-04 06:01:26,088 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 06:01:26,088 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 06:01:26,088 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:01:26,088 INFO [Listener at localhost/35025] procedure2.ProcedureExecutor(628): Stopping 2023-09-04 06:01:26,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:01:26,089 DEBUG [Listener at localhost/35025] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7d2b5906 to 127.0.0.1:49156 2023-09-04 06:01:26,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:01:26,089 DEBUG [Listener at localhost/35025] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:01:26,089 INFO [Listener at localhost/35025] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,37843,1693807224102' ***** 2023-09-04 06:01:26,089 INFO [Listener at localhost/35025] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-09-04 06:01:26,089 INFO [RS:0;jenkins-hbase4:37843] regionserver.HeapMemoryManager(220): Stopping 2023-09-04 06:01:26,090 INFO [RS:0;jenkins-hbase4:37843] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-09-04 06:01:26,090 INFO [RS:0;jenkins-hbase4:37843] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-09-04 06:01:26,090 INFO [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(3305): Received CLOSE for 96ba21e9bc791d113f0f7f6ea8c0e0a6 2023-09-04 06:01:26,090 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-09-04 06:01:26,090 INFO [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(3305): Received CLOSE for 47540753994bd60ea56f493ca9b5ce6f 2023-09-04 06:01:26,090 INFO [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:26,090 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 96ba21e9bc791d113f0f7f6ea8c0e0a6, disabling compactions & flushes 2023-09-04 06:01:26,090 DEBUG [RS:0;jenkins-hbase4:37843] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6a10a7d1 to 127.0.0.1:49156 2023-09-04 06:01:26,090 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:01:26,090 DEBUG [RS:0;jenkins-hbase4:37843] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:01:26,090 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:01:26,090 INFO [RS:0;jenkins-hbase4:37843] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-09-04 06:01:26,090 INFO [RS:0;jenkins-hbase4:37843] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-09-04 06:01:26,091 INFO [RS:0;jenkins-hbase4:37843] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-09-04 06:01:26,090 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. after waiting 0 ms 2023-09-04 06:01:26,091 INFO [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-09-04 06:01:26,091 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:01:26,091 INFO [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-09-04 06:01:26,091 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 96ba21e9bc791d113f0f7f6ea8c0e0a6 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-04 06:01:26,091 DEBUG [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(1478): Online Regions={96ba21e9bc791d113f0f7f6ea8c0e0a6=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6., 1588230740=hbase:meta,,1.1588230740, 47540753994bd60ea56f493ca9b5ce6f=hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f.} 2023-09-04 06:01:26,091 DEBUG [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(1504): Waiting on 1588230740, 47540753994bd60ea56f493ca9b5ce6f, 96ba21e9bc791d113f0f7f6ea8c0e0a6 2023-09-04 06:01:26,094 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 06:01:26,094 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 06:01:26,094 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 06:01:26,094 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 06:01:26,094 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 06:01:26,094 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.10 KB heapSize=5.61 KB 2023-09-04 06:01:26,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=22 (bloomFilter=true), to=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/.tmp/info/7407f934d5d0491cb1c0df9d742dcb7e 2023-09-04 06:01:26,130 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.84 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/.tmp/info/338721ec8b93441abaaaf89d8a55008c 2023-09-04 06:01:26,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/.tmp/info/7407f934d5d0491cb1c0df9d742dcb7e as hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/7407f934d5d0491cb1c0df9d742dcb7e 2023-09-04 06:01:26,142 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/7407f934d5d0491cb1c0df9d742dcb7e, entries=1, sequenceid=22, filesize=5.8 K 2023-09-04 06:01:26,144 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 96ba21e9bc791d113f0f7f6ea8c0e0a6 in 52ms, sequenceid=22, compaction requested=true 2023-09-04 06:01:26,151 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/063538088e764474947ba3a3b145c48a, hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/a29b3883ef4d4b08809b39a63b8047ff, hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/4d7935d37f184b179a1b46c2388b00aa] to archive 2023-09-04 06:01:26,154 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-09-04 06:01:26,160 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/063538088e764474947ba3a3b145c48a to hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/063538088e764474947ba3a3b145c48a 2023-09-04 06:01:26,160 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=264 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/.tmp/table/f0e734800e8446bd85ff2e783f7d7ecf 2023-09-04 06:01:26,162 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/a29b3883ef4d4b08809b39a63b8047ff to hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/a29b3883ef4d4b08809b39a63b8047ff 2023-09-04 06:01:26,164 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/4d7935d37f184b179a1b46c2388b00aa to hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/info/4d7935d37f184b179a1b46c2388b00aa 2023-09-04 06:01:26,169 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/.tmp/info/338721ec8b93441abaaaf89d8a55008c as hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/info/338721ec8b93441abaaaf89d8a55008c 2023-09-04 06:01:26,178 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/info/338721ec8b93441abaaaf89d8a55008c, entries=20, sequenceid=14, filesize=7.6 K 2023-09-04 06:01:26,179 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/.tmp/table/f0e734800e8446bd85ff2e783f7d7ecf as hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/table/f0e734800e8446bd85ff2e783f7d7ecf 2023-09-04 06:01:26,186 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/table/f0e734800e8446bd85ff2e783f7d7ecf, entries=4, sequenceid=14, filesize=4.9 K 2023-09-04 06:01:26,187 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.10 KB/3174, heapSize ~5.33 KB/5456, currentSize=0 B/0 for 1588230740 in 93ms, sequenceid=14, compaction requested=false 2023-09-04 06:01:26,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/96ba21e9bc791d113f0f7f6ea8c0e0a6/recovered.edits/25.seqid, newMaxSeqId=25, maxSeqId=1 2023-09-04 06:01:26,200 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:01:26,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 96ba21e9bc791d113f0f7f6ea8c0e0a6: 2023-09-04 06:01:26,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693807225465.96ba21e9bc791d113f0f7f6ea8c0e0a6. 2023-09-04 06:01:26,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 47540753994bd60ea56f493ca9b5ce6f, disabling compactions & flushes 2023-09-04 06:01:26,200 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. 2023-09-04 06:01:26,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. 2023-09-04 06:01:26,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. after waiting 0 ms 2023-09-04 06:01:26,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. 2023-09-04 06:01:26,202 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-09-04 06:01:26,203 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-04 06:01:26,204 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 06:01:26,204 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 06:01:26,204 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-09-04 06:01:26,206 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/data/hbase/namespace/47540753994bd60ea56f493ca9b5ce6f/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-09-04 06:01:26,207 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. 2023-09-04 06:01:26,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 47540753994bd60ea56f493ca9b5ce6f: 2023-09-04 06:01:26,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693807224863.47540753994bd60ea56f493ca9b5ce6f. 2023-09-04 06:01:26,291 INFO [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,37843,1693807224102; all regions closed. 2023-09-04 06:01:26,298 DEBUG [RS:0;jenkins-hbase4:37843] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/oldWALs 2023-09-04 06:01:26,298 INFO [RS:0;jenkins-hbase4:37843] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37843%2C1693807224102.meta:.meta(num 1693807224792) 2023-09-04 06:01:26,304 DEBUG [RS:0;jenkins-hbase4:37843] wal.AbstractFSWAL(1028): Moved 2 WAL file(s) to /user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/oldWALs 2023-09-04 06:01:26,304 INFO [RS:0;jenkins-hbase4:37843] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37843%2C1693807224102:(num 1693807286066) 2023-09-04 06:01:26,304 DEBUG [RS:0;jenkins-hbase4:37843] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:01:26,304 INFO [RS:0;jenkins-hbase4:37843] regionserver.LeaseManager(133): Closed leases 2023-09-04 06:01:26,304 INFO [RS:0;jenkins-hbase4:37843] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-09-04 06:01:26,304 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 06:01:26,305 INFO [RS:0;jenkins-hbase4:37843] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37843 2023-09-04 06:01:26,307 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37843,1693807224102 2023-09-04 06:01:26,307 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 06:01:26,308 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 06:01:26,309 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,37843,1693807224102] 2023-09-04 06:01:26,309 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,37843,1693807224102; numProcessing=1 2023-09-04 06:01:26,310 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,37843,1693807224102 already deleted, retry=false 2023-09-04 06:01:26,310 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,37843,1693807224102 expired; onlineServers=0 2023-09-04 06:01:26,310 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38737,1693807224054' ***** 2023-09-04 06:01:26,310 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-09-04 06:01:26,310 DEBUG [M:0;jenkins-hbase4:38737] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@297b2684, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 06:01:26,311 INFO [M:0;jenkins-hbase4:38737] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38737,1693807224054 2023-09-04 06:01:26,311 INFO [M:0;jenkins-hbase4:38737] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38737,1693807224054; all regions closed. 2023-09-04 06:01:26,311 DEBUG [M:0;jenkins-hbase4:38737] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:01:26,311 DEBUG [M:0;jenkins-hbase4:38737] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-09-04 06:01:26,311 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-09-04 06:01:26,311 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693807224381] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693807224381,5,FailOnTimeoutGroup] 2023-09-04 06:01:26,311 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693807224381] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693807224381,5,FailOnTimeoutGroup] 2023-09-04 06:01:26,311 DEBUG [M:0;jenkins-hbase4:38737] cleaner.HFileCleaner(317): Stopping file delete threads 2023-09-04 06:01:26,312 INFO [M:0;jenkins-hbase4:38737] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-09-04 06:01:26,312 INFO [M:0;jenkins-hbase4:38737] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-09-04 06:01:26,312 INFO [M:0;jenkins-hbase4:38737] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-09-04 06:01:26,312 DEBUG [M:0;jenkins-hbase4:38737] master.HMaster(1512): Stopping service threads 2023-09-04 06:01:26,312 INFO [M:0;jenkins-hbase4:38737] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-09-04 06:01:26,312 ERROR [M:0;jenkins-hbase4:38737] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-09-04 06:01:26,312 INFO [M:0;jenkins-hbase4:38737] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-09-04 06:01:26,312 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-09-04 06:01:26,313 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-09-04 06:01:26,313 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:01:26,313 DEBUG [M:0;jenkins-hbase4:38737] zookeeper.ZKUtil(398): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-09-04 06:01:26,313 WARN [M:0;jenkins-hbase4:38737] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-09-04 06:01:26,313 INFO [M:0;jenkins-hbase4:38737] assignment.AssignmentManager(315): Stopping assignment manager 2023-09-04 06:01:26,313 INFO [M:0;jenkins-hbase4:38737] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-09-04 06:01:26,314 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 06:01:26,314 DEBUG [M:0;jenkins-hbase4:38737] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 06:01:26,314 INFO [M:0;jenkins-hbase4:38737] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:01:26,314 DEBUG [M:0;jenkins-hbase4:38737] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:01:26,314 DEBUG [M:0;jenkins-hbase4:38737] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 06:01:26,314 DEBUG [M:0;jenkins-hbase4:38737] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:01:26,314 INFO [M:0;jenkins-hbase4:38737] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.89 KB heapSize=47.33 KB 2023-09-04 06:01:26,327 INFO [M:0;jenkins-hbase4:38737] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.89 KB at sequenceid=100 (bloomFilter=true), to=hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/8e46163227e149c8bef449327460b5f1 2023-09-04 06:01:26,333 INFO [M:0;jenkins-hbase4:38737] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8e46163227e149c8bef449327460b5f1 2023-09-04 06:01:26,334 DEBUG [M:0;jenkins-hbase4:38737] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/8e46163227e149c8bef449327460b5f1 as hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/8e46163227e149c8bef449327460b5f1 2023-09-04 06:01:26,340 INFO [M:0;jenkins-hbase4:38737] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8e46163227e149c8bef449327460b5f1 2023-09-04 06:01:26,340 INFO [M:0;jenkins-hbase4:38737] regionserver.HStore(1080): Added hdfs://localhost:43627/user/jenkins/test-data/340a08de-9379-729a-00ba-42ca833d1ad3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/8e46163227e149c8bef449327460b5f1, entries=11, sequenceid=100, filesize=6.1 K 2023-09-04 06:01:26,341 INFO [M:0;jenkins-hbase4:38737] regionserver.HRegion(2948): Finished flush of dataSize ~38.89 KB/39824, heapSize ~47.31 KB/48448, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 27ms, sequenceid=100, compaction requested=false 2023-09-04 06:01:26,343 INFO [M:0;jenkins-hbase4:38737] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:01:26,343 DEBUG [M:0;jenkins-hbase4:38737] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 06:01:26,346 INFO [M:0;jenkins-hbase4:38737] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-09-04 06:01:26,346 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 06:01:26,346 INFO [M:0;jenkins-hbase4:38737] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38737 2023-09-04 06:01:26,348 DEBUG [M:0;jenkins-hbase4:38737] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,38737,1693807224054 already deleted, retry=false 2023-09-04 06:01:26,409 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:01:26,409 INFO [RS:0;jenkins-hbase4:37843] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,37843,1693807224102; zookeeper connection closed. 2023-09-04 06:01:26,409 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): regionserver:37843-0x10075865c870001, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:01:26,410 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@90d3873] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@90d3873 2023-09-04 06:01:26,410 INFO [Listener at localhost/35025] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-09-04 06:01:26,500 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-09-04 06:01:26,509 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:01:26,509 INFO [M:0;jenkins-hbase4:38737] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38737,1693807224054; zookeeper connection closed. 2023-09-04 06:01:26,510 DEBUG [Listener at localhost/35025-EventThread] zookeeper.ZKWatcher(600): master:38737-0x10075865c870000, quorum=127.0.0.1:49156, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:01:26,510 WARN [Listener at localhost/35025] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 06:01:26,514 INFO [Listener at localhost/35025] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:01:26,618 WARN [BP-1642889630-172.31.14.131-1693807223466 heartbeating to localhost/127.0.0.1:43627] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 06:01:26,618 WARN [BP-1642889630-172.31.14.131-1693807223466 heartbeating to localhost/127.0.0.1:43627] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1642889630-172.31.14.131-1693807223466 (Datanode Uuid e33e49c9-c2d4-43ae-9334-4f1ff4cfb445) service to localhost/127.0.0.1:43627 2023-09-04 06:01:26,619 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/cluster_406daa38-4a7a-08be-d885-7f6397132708/dfs/data/data3/current/BP-1642889630-172.31.14.131-1693807223466] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:01:26,619 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/cluster_406daa38-4a7a-08be-d885-7f6397132708/dfs/data/data4/current/BP-1642889630-172.31.14.131-1693807223466] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:01:26,621 WARN [Listener at localhost/35025] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 06:01:26,623 INFO [Listener at localhost/35025] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:01:26,727 WARN [BP-1642889630-172.31.14.131-1693807223466 heartbeating to localhost/127.0.0.1:43627] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 06:01:26,727 WARN [BP-1642889630-172.31.14.131-1693807223466 heartbeating to localhost/127.0.0.1:43627] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1642889630-172.31.14.131-1693807223466 (Datanode Uuid e7691a50-e56f-4eac-bd8e-c4481fb69549) service to localhost/127.0.0.1:43627 2023-09-04 06:01:26,728 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/cluster_406daa38-4a7a-08be-d885-7f6397132708/dfs/data/data1/current/BP-1642889630-172.31.14.131-1693807223466] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:01:26,728 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/cluster_406daa38-4a7a-08be-d885-7f6397132708/dfs/data/data2/current/BP-1642889630-172.31.14.131-1693807223466] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:01:26,739 INFO [Listener at localhost/35025] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:01:26,852 INFO [Listener at localhost/35025] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-09-04 06:01:26,868 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-09-04 06:01:26,878 INFO [Listener at localhost/35025] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=63 (was 78), OpenFileDescriptor=440 (was 467), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=71 (was 144), ProcessCount=166 (was 166), AvailableMemoryMB=8258 (was 8493) 2023-09-04 06:01:26,887 INFO [Listener at localhost/35025] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=64, OpenFileDescriptor=440, MaxFileDescriptor=60000, SystemLoadAverage=71, ProcessCount=166, AvailableMemoryMB=8258 2023-09-04 06:01:26,887 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-09-04 06:01:26,888 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/hadoop.log.dir so I do NOT create it in target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83 2023-09-04 06:01:26,888 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5e04e905-e58c-993d-1199-718cbb4d90e5/hadoop.tmp.dir so I do NOT create it in target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83 2023-09-04 06:01:26,888 INFO [Listener at localhost/35025] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/cluster_200fa038-4496-7b8d-6264-2148931f885c, deleteOnExit=true 2023-09-04 06:01:26,888 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-09-04 06:01:26,888 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/test.cache.data in system properties and HBase conf 2023-09-04 06:01:26,888 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/hadoop.tmp.dir in system properties and HBase conf 2023-09-04 06:01:26,888 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/hadoop.log.dir in system properties and HBase conf 2023-09-04 06:01:26,888 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/mapreduce.cluster.local.dir in system properties and HBase conf 2023-09-04 06:01:26,888 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-09-04 06:01:26,888 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-09-04 06:01:26,889 DEBUG [Listener at localhost/35025] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-09-04 06:01:26,889 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-09-04 06:01:26,889 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-09-04 06:01:26,889 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-09-04 06:01:26,889 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 06:01:26,889 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-09-04 06:01:26,890 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-09-04 06:01:26,890 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 06:01:26,890 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 06:01:26,890 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-09-04 06:01:26,890 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/nfs.dump.dir in system properties and HBase conf 2023-09-04 06:01:26,890 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/java.io.tmpdir in system properties and HBase conf 2023-09-04 06:01:26,890 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 06:01:26,890 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-09-04 06:01:26,890 INFO [Listener at localhost/35025] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-09-04 06:01:26,895 WARN [Listener at localhost/35025] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 06:01:26,895 WARN [Listener at localhost/35025] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 06:01:26,945 WARN [Listener at localhost/35025] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 06:01:26,947 INFO [Listener at localhost/35025] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 06:01:26,953 INFO [Listener at localhost/35025] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/java.io.tmpdir/Jetty_localhost_34577_hdfs____wijon2/webapp 2023-09-04 06:01:27,045 INFO [Listener at localhost/35025] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34577 2023-09-04 06:01:27,049 WARN [Listener at localhost/35025] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 06:01:27,049 WARN [Listener at localhost/35025] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 06:01:27,093 WARN [Listener at localhost/38163] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 06:01:27,104 WARN [Listener at localhost/38163] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 06:01:27,106 WARN [Listener at localhost/38163] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 06:01:27,107 INFO [Listener at localhost/38163] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 06:01:27,111 INFO [Listener at localhost/38163] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/java.io.tmpdir/Jetty_localhost_33059_datanode____m0i7a3/webapp 2023-09-04 06:01:27,203 INFO [Listener at localhost/38163] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33059 2023-09-04 06:01:27,209 WARN [Listener at localhost/42255] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 06:01:27,220 WARN [Listener at localhost/42255] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 06:01:27,222 WARN [Listener at localhost/42255] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 06:01:27,224 INFO [Listener at localhost/42255] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 06:01:27,227 INFO [Listener at localhost/42255] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/java.io.tmpdir/Jetty_localhost_35669_datanode____4apkxq/webapp 2023-09-04 06:01:27,303 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe553fa2a6a7eca28: Processing first storage report for DS-d3fb595d-6b6c-4a03-a377-2f38ea57a3d3 from datanode 14656170-88c6-4379-82a4-7527f7d4d25f 2023-09-04 06:01:27,303 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe553fa2a6a7eca28: from storage DS-d3fb595d-6b6c-4a03-a377-2f38ea57a3d3 node DatanodeRegistration(127.0.0.1:37037, datanodeUuid=14656170-88c6-4379-82a4-7527f7d4d25f, infoPort=33233, infoSecurePort=0, ipcPort=42255, storageInfo=lv=-57;cid=testClusterID;nsid=744585134;c=1693807286898), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:01:27,303 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe553fa2a6a7eca28: Processing first storage report for DS-3ec710b1-8956-4b58-ad0d-1f54b3df6dc8 from datanode 14656170-88c6-4379-82a4-7527f7d4d25f 2023-09-04 06:01:27,303 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe553fa2a6a7eca28: from storage DS-3ec710b1-8956-4b58-ad0d-1f54b3df6dc8 node DatanodeRegistration(127.0.0.1:37037, datanodeUuid=14656170-88c6-4379-82a4-7527f7d4d25f, infoPort=33233, infoSecurePort=0, ipcPort=42255, storageInfo=lv=-57;cid=testClusterID;nsid=744585134;c=1693807286898), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:01:27,322 INFO [Listener at localhost/42255] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35669 2023-09-04 06:01:27,329 WARN [Listener at localhost/37417] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 06:01:27,428 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xefe1ef9f403e1475: Processing first storage report for DS-34087bb1-246a-42ca-b408-b7b12824778b from datanode f79419e4-9101-4ccb-92e1-69994ef4a42e 2023-09-04 06:01:27,428 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xefe1ef9f403e1475: from storage DS-34087bb1-246a-42ca-b408-b7b12824778b node DatanodeRegistration(127.0.0.1:43733, datanodeUuid=f79419e4-9101-4ccb-92e1-69994ef4a42e, infoPort=33519, infoSecurePort=0, ipcPort=37417, storageInfo=lv=-57;cid=testClusterID;nsid=744585134;c=1693807286898), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-09-04 06:01:27,428 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xefe1ef9f403e1475: Processing first storage report for DS-1b61e401-001f-4bc8-9caa-7c0c769fa814 from datanode f79419e4-9101-4ccb-92e1-69994ef4a42e 2023-09-04 06:01:27,428 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xefe1ef9f403e1475: from storage DS-1b61e401-001f-4bc8-9caa-7c0c769fa814 node DatanodeRegistration(127.0.0.1:43733, datanodeUuid=f79419e4-9101-4ccb-92e1-69994ef4a42e, infoPort=33519, infoSecurePort=0, ipcPort=37417, storageInfo=lv=-57;cid=testClusterID;nsid=744585134;c=1693807286898), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:01:27,438 DEBUG [Listener at localhost/37417] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83 2023-09-04 06:01:27,440 INFO [Listener at localhost/37417] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/cluster_200fa038-4496-7b8d-6264-2148931f885c/zookeeper_0, clientPort=56951, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/cluster_200fa038-4496-7b8d-6264-2148931f885c/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/cluster_200fa038-4496-7b8d-6264-2148931f885c/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-09-04 06:01:27,442 INFO [Listener at localhost/37417] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=56951 2023-09-04 06:01:27,442 INFO [Listener at localhost/37417] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:01:27,443 INFO [Listener at localhost/37417] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:01:27,457 INFO [Listener at localhost/37417] util.FSUtils(471): Created version file at hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058 with version=8 2023-09-04 06:01:27,457 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/hbase-staging 2023-09-04 06:01:27,459 INFO [Listener at localhost/37417] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 06:01:27,459 INFO [Listener at localhost/37417] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:01:27,459 INFO [Listener at localhost/37417] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 06:01:27,459 INFO [Listener at localhost/37417] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 06:01:27,460 INFO [Listener at localhost/37417] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:01:27,460 INFO [Listener at localhost/37417] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 06:01:27,460 INFO [Listener at localhost/37417] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 06:01:27,461 INFO [Listener at localhost/37417] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46229 2023-09-04 06:01:27,461 INFO [Listener at localhost/37417] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:01:27,462 INFO [Listener at localhost/37417] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:01:27,463 INFO [Listener at localhost/37417] zookeeper.RecoverableZooKeeper(93): Process identifier=master:46229 connecting to ZooKeeper ensemble=127.0.0.1:56951 2023-09-04 06:01:27,469 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:462290x0, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 06:01:27,470 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:46229-0x100758754360000 connected 2023-09-04 06:01:27,485 DEBUG [Listener at localhost/37417] zookeeper.ZKUtil(164): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 06:01:27,485 DEBUG [Listener at localhost/37417] zookeeper.ZKUtil(164): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:01:27,486 DEBUG [Listener at localhost/37417] zookeeper.ZKUtil(164): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 06:01:27,486 DEBUG [Listener at localhost/37417] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46229 2023-09-04 06:01:27,487 DEBUG [Listener at localhost/37417] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46229 2023-09-04 06:01:27,487 DEBUG [Listener at localhost/37417] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46229 2023-09-04 06:01:27,488 DEBUG [Listener at localhost/37417] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46229 2023-09-04 06:01:27,488 DEBUG [Listener at localhost/37417] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46229 2023-09-04 06:01:27,488 INFO [Listener at localhost/37417] master.HMaster(444): hbase.rootdir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058, hbase.cluster.distributed=false 2023-09-04 06:01:27,501 INFO [Listener at localhost/37417] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 06:01:27,501 INFO [Listener at localhost/37417] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:01:27,501 INFO [Listener at localhost/37417] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 06:01:27,501 INFO [Listener at localhost/37417] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 06:01:27,502 INFO [Listener at localhost/37417] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:01:27,502 INFO [Listener at localhost/37417] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 06:01:27,502 INFO [Listener at localhost/37417] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 06:01:27,503 INFO [Listener at localhost/37417] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44413 2023-09-04 06:01:27,503 INFO [Listener at localhost/37417] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-04 06:01:27,504 DEBUG [Listener at localhost/37417] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-09-04 06:01:27,505 INFO [Listener at localhost/37417] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:01:27,506 INFO [Listener at localhost/37417] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:01:27,507 INFO [Listener at localhost/37417] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:44413 connecting to ZooKeeper ensemble=127.0.0.1:56951 2023-09-04 06:01:27,510 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): regionserver:444130x0, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 06:01:27,510 DEBUG [Listener at localhost/37417] zookeeper.ZKUtil(164): regionserver:444130x0, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 06:01:27,511 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:44413-0x100758754360001 connected 2023-09-04 06:01:27,511 DEBUG [Listener at localhost/37417] zookeeper.ZKUtil(164): regionserver:44413-0x100758754360001, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:01:27,512 DEBUG [Listener at localhost/37417] zookeeper.ZKUtil(164): regionserver:44413-0x100758754360001, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 06:01:27,512 DEBUG [Listener at localhost/37417] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44413 2023-09-04 06:01:27,512 DEBUG [Listener at localhost/37417] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44413 2023-09-04 06:01:27,512 DEBUG [Listener at localhost/37417] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44413 2023-09-04 06:01:27,513 DEBUG [Listener at localhost/37417] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44413 2023-09-04 06:01:27,513 DEBUG [Listener at localhost/37417] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44413 2023-09-04 06:01:27,514 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,46229,1693807287459 2023-09-04 06:01:27,517 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 06:01:27,517 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,46229,1693807287459 2023-09-04 06:01:27,518 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 06:01:27,518 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): regionserver:44413-0x100758754360001, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 06:01:27,519 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:01:27,519 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 06:01:27,520 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,46229,1693807287459 from backup master directory 2023-09-04 06:01:27,520 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 06:01:27,521 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,46229,1693807287459 2023-09-04 06:01:27,521 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 06:01:27,521 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 06:01:27,521 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,46229,1693807287459 2023-09-04 06:01:27,535 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/hbase.id with ID: ada0a373-a373-4229-a67e-3e8b5245ef3d 2023-09-04 06:01:27,548 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:01:27,551 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:01:27,562 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x467c2c8c to 127.0.0.1:56951 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 06:01:27,566 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@60164a7f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 06:01:27,566 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-04 06:01:27,567 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-09-04 06:01:27,567 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 06:01:27,568 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/MasterData/data/master/store-tmp 2023-09-04 06:01:27,579 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:01:27,579 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 06:01:27,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:01:27,579 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:01:27,579 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 06:01:27,579 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:01:27,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:01:27,579 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 06:01:27,580 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/MasterData/WALs/jenkins-hbase4.apache.org,46229,1693807287459 2023-09-04 06:01:27,584 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46229%2C1693807287459, suffix=, logDir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/MasterData/WALs/jenkins-hbase4.apache.org,46229,1693807287459, archiveDir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/MasterData/oldWALs, maxLogs=10 2023-09-04 06:01:27,605 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37037,DS-d3fb595d-6b6c-4a03-a377-2f38ea57a3d3,DISK] 2023-09-04 06:01:27,606 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43733,DS-34087bb1-246a-42ca-b408-b7b12824778b,DISK] 2023-09-04 06:01:27,608 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/MasterData/WALs/jenkins-hbase4.apache.org,46229,1693807287459/jenkins-hbase4.apache.org%2C46229%2C1693807287459.1693807287584 2023-09-04 06:01:27,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37037,DS-d3fb595d-6b6c-4a03-a377-2f38ea57a3d3,DISK], DatanodeInfoWithStorage[127.0.0.1:43733,DS-34087bb1-246a-42ca-b408-b7b12824778b,DISK]] 2023-09-04 06:01:27,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-09-04 06:01:27,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:01:27,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:01:27,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:01:27,611 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:01:27,613 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-09-04 06:01:27,613 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-09-04 06:01:27,614 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:01:27,614 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:01:27,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:01:27,618 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:01:27,620 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 06:01:27,620 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=793437, jitterRate=0.008908465504646301}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 06:01:27,620 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 06:01:27,621 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-09-04 06:01:27,622 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-09-04 06:01:27,622 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-09-04 06:01:27,622 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-09-04 06:01:27,622 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-09-04 06:01:27,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 0 msec 2023-09-04 06:01:27,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-09-04 06:01:27,623 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-09-04 06:01:27,624 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-09-04 06:01:27,635 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-09-04 06:01:27,635 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-09-04 06:01:27,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-09-04 06:01:27,636 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-09-04 06:01:27,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-09-04 06:01:27,638 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:01:27,639 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-09-04 06:01:27,639 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-09-04 06:01:27,640 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-09-04 06:01:27,642 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 06:01:27,642 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): regionserver:44413-0x100758754360001, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 06:01:27,642 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:01:27,642 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,46229,1693807287459, sessionid=0x100758754360000, setting cluster-up flag (Was=false) 2023-09-04 06:01:27,646 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:01:27,650 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-09-04 06:01:27,650 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46229,1693807287459 2023-09-04 06:01:27,654 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:01:27,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-09-04 06:01:27,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46229,1693807287459 2023-09-04 06:01:27,659 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/.hbase-snapshot/.tmp 2023-09-04 06:01:27,661 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-09-04 06:01:27,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:01:27,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:01:27,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:01:27,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:01:27,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-09-04 06:01:27,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:01:27,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 06:01:27,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:01:27,663 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693807317663 2023-09-04 06:01:27,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-09-04 06:01:27,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-09-04 06:01:27,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-09-04 06:01:27,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-09-04 06:01:27,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-09-04 06:01:27,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-09-04 06:01:27,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:27,665 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 06:01:27,665 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-09-04 06:01:27,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-09-04 06:01:27,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-09-04 06:01:27,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-09-04 06:01:27,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-09-04 06:01:27,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-09-04 06:01:27,666 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693807287666,5,FailOnTimeoutGroup] 2023-09-04 06:01:27,667 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693807287667,5,FailOnTimeoutGroup] 2023-09-04 06:01:27,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:27,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-09-04 06:01:27,667 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 06:01:27,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:27,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:27,678 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 06:01:27,679 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 06:01:27,680 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058 2023-09-04 06:01:27,690 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:01:27,691 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 06:01:27,693 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/info 2023-09-04 06:01:27,694 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 06:01:27,694 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:01:27,694 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 06:01:27,696 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/rep_barrier 2023-09-04 06:01:27,696 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 06:01:27,696 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:01:27,697 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 06:01:27,698 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/table 2023-09-04 06:01:27,698 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 06:01:27,698 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:01:27,699 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740 2023-09-04 06:01:27,700 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740 2023-09-04 06:01:27,702 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 06:01:27,703 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 06:01:27,706 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 06:01:27,707 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=813772, jitterRate=0.0347653329372406}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 06:01:27,707 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 06:01:27,707 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 06:01:27,707 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 06:01:27,707 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 06:01:27,707 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 06:01:27,707 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 06:01:27,707 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 06:01:27,707 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 06:01:27,709 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 06:01:27,709 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-09-04 06:01:27,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-09-04 06:01:27,710 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-09-04 06:01:27,712 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-09-04 06:01:27,715 INFO [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(951): ClusterId : ada0a373-a373-4229-a67e-3e8b5245ef3d 2023-09-04 06:01:27,715 DEBUG [RS:0;jenkins-hbase4:44413] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-09-04 06:01:27,717 DEBUG [RS:0;jenkins-hbase4:44413] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-09-04 06:01:27,717 DEBUG [RS:0;jenkins-hbase4:44413] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-09-04 06:01:27,719 DEBUG [RS:0;jenkins-hbase4:44413] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-09-04 06:01:27,720 DEBUG [RS:0;jenkins-hbase4:44413] zookeeper.ReadOnlyZKClient(139): Connect 0x60f1984e to 127.0.0.1:56951 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 06:01:27,723 DEBUG [RS:0;jenkins-hbase4:44413] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6e366a20, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 06:01:27,723 DEBUG [RS:0;jenkins-hbase4:44413] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6a09f33e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 06:01:27,732 DEBUG [RS:0;jenkins-hbase4:44413] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:44413 2023-09-04 06:01:27,732 INFO [RS:0;jenkins-hbase4:44413] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-09-04 06:01:27,732 INFO [RS:0;jenkins-hbase4:44413] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-09-04 06:01:27,732 DEBUG [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(1022): About to register with Master. 2023-09-04 06:01:27,733 INFO [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,46229,1693807287459 with isa=jenkins-hbase4.apache.org/172.31.14.131:44413, startcode=1693807287501 2023-09-04 06:01:27,733 DEBUG [RS:0;jenkins-hbase4:44413] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-09-04 06:01:27,736 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33525, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-09-04 06:01:27,737 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=46229] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:01:27,737 DEBUG [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058 2023-09-04 06:01:27,737 DEBUG [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:38163 2023-09-04 06:01:27,737 DEBUG [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-09-04 06:01:27,739 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 06:01:27,740 DEBUG [RS:0;jenkins-hbase4:44413] zookeeper.ZKUtil(162): regionserver:44413-0x100758754360001, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:01:27,740 WARN [RS:0;jenkins-hbase4:44413] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 06:01:27,740 INFO [RS:0;jenkins-hbase4:44413] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 06:01:27,740 DEBUG [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(1948): logDir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/WALs/jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:01:27,740 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,44413,1693807287501] 2023-09-04 06:01:27,744 DEBUG [RS:0;jenkins-hbase4:44413] zookeeper.ZKUtil(162): regionserver:44413-0x100758754360001, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:01:27,745 DEBUG [RS:0;jenkins-hbase4:44413] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-09-04 06:01:27,745 INFO [RS:0;jenkins-hbase4:44413] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-09-04 06:01:27,746 INFO [RS:0;jenkins-hbase4:44413] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-09-04 06:01:27,746 INFO [RS:0;jenkins-hbase4:44413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-09-04 06:01:27,746 INFO [RS:0;jenkins-hbase4:44413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:27,748 INFO [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-09-04 06:01:27,749 INFO [RS:0;jenkins-hbase4:44413] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:27,749 DEBUG [RS:0;jenkins-hbase4:44413] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:01:27,750 DEBUG [RS:0;jenkins-hbase4:44413] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:01:27,750 DEBUG [RS:0;jenkins-hbase4:44413] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:01:27,750 DEBUG [RS:0;jenkins-hbase4:44413] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:01:27,750 DEBUG [RS:0;jenkins-hbase4:44413] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:01:27,750 DEBUG [RS:0;jenkins-hbase4:44413] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 06:01:27,750 DEBUG [RS:0;jenkins-hbase4:44413] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:01:27,750 DEBUG [RS:0;jenkins-hbase4:44413] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:01:27,750 DEBUG [RS:0;jenkins-hbase4:44413] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:01:27,750 DEBUG [RS:0;jenkins-hbase4:44413] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:01:27,751 INFO [RS:0;jenkins-hbase4:44413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:27,751 INFO [RS:0;jenkins-hbase4:44413] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:27,751 INFO [RS:0;jenkins-hbase4:44413] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:27,764 INFO [RS:0;jenkins-hbase4:44413] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-09-04 06:01:27,765 INFO [RS:0;jenkins-hbase4:44413] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44413,1693807287501-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:27,777 INFO [RS:0;jenkins-hbase4:44413] regionserver.Replication(203): jenkins-hbase4.apache.org,44413,1693807287501 started 2023-09-04 06:01:27,777 INFO [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,44413,1693807287501, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:44413, sessionid=0x100758754360001 2023-09-04 06:01:27,777 DEBUG [RS:0;jenkins-hbase4:44413] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-09-04 06:01:27,778 DEBUG [RS:0;jenkins-hbase4:44413] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:01:27,778 DEBUG [RS:0;jenkins-hbase4:44413] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44413,1693807287501' 2023-09-04 06:01:27,778 DEBUG [RS:0;jenkins-hbase4:44413] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 06:01:27,778 DEBUG [RS:0;jenkins-hbase4:44413] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 06:01:27,779 DEBUG [RS:0;jenkins-hbase4:44413] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-09-04 06:01:27,779 DEBUG [RS:0;jenkins-hbase4:44413] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-09-04 06:01:27,779 DEBUG [RS:0;jenkins-hbase4:44413] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:01:27,779 DEBUG [RS:0;jenkins-hbase4:44413] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44413,1693807287501' 2023-09-04 06:01:27,779 DEBUG [RS:0;jenkins-hbase4:44413] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-09-04 06:01:27,779 DEBUG [RS:0;jenkins-hbase4:44413] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-09-04 06:01:27,780 DEBUG [RS:0;jenkins-hbase4:44413] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-09-04 06:01:27,780 INFO [RS:0;jenkins-hbase4:44413] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-09-04 06:01:27,780 INFO [RS:0;jenkins-hbase4:44413] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-09-04 06:01:27,862 DEBUG [jenkins-hbase4:46229] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-09-04 06:01:27,863 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,44413,1693807287501, state=OPENING 2023-09-04 06:01:27,864 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-09-04 06:01:27,865 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:01:27,866 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,44413,1693807287501}] 2023-09-04 06:01:27,866 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 06:01:27,882 INFO [RS:0;jenkins-hbase4:44413] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44413%2C1693807287501, suffix=, logDir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/WALs/jenkins-hbase4.apache.org,44413,1693807287501, archiveDir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/oldWALs, maxLogs=32 2023-09-04 06:01:27,899 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43733,DS-34087bb1-246a-42ca-b408-b7b12824778b,DISK] 2023-09-04 06:01:27,900 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37037,DS-d3fb595d-6b6c-4a03-a377-2f38ea57a3d3,DISK] 2023-09-04 06:01:27,902 INFO [RS:0;jenkins-hbase4:44413] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/WALs/jenkins-hbase4.apache.org,44413,1693807287501/jenkins-hbase4.apache.org%2C44413%2C1693807287501.1693807287883 2023-09-04 06:01:27,902 DEBUG [RS:0;jenkins-hbase4:44413] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43733,DS-34087bb1-246a-42ca-b408-b7b12824778b,DISK], DatanodeInfoWithStorage[127.0.0.1:37037,DS-d3fb595d-6b6c-4a03-a377-2f38ea57a3d3,DISK]] 2023-09-04 06:01:28,020 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:01:28,021 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-09-04 06:01:28,023 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48610, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-09-04 06:01:28,027 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-09-04 06:01:28,027 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 06:01:28,029 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44413%2C1693807287501.meta, suffix=.meta, logDir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/WALs/jenkins-hbase4.apache.org,44413,1693807287501, archiveDir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/oldWALs, maxLogs=32 2023-09-04 06:01:28,044 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37037,DS-d3fb595d-6b6c-4a03-a377-2f38ea57a3d3,DISK] 2023-09-04 06:01:28,045 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43733,DS-34087bb1-246a-42ca-b408-b7b12824778b,DISK] 2023-09-04 06:01:28,046 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/WALs/jenkins-hbase4.apache.org,44413,1693807287501/jenkins-hbase4.apache.org%2C44413%2C1693807287501.meta.1693807288030.meta 2023-09-04 06:01:28,047 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37037,DS-d3fb595d-6b6c-4a03-a377-2f38ea57a3d3,DISK], DatanodeInfoWithStorage[127.0.0.1:43733,DS-34087bb1-246a-42ca-b408-b7b12824778b,DISK]] 2023-09-04 06:01:28,047 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-09-04 06:01:28,047 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-09-04 06:01:28,047 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-09-04 06:01:28,047 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-09-04 06:01:28,047 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-09-04 06:01:28,047 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:01:28,047 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-09-04 06:01:28,047 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-09-04 06:01:28,049 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 06:01:28,050 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/info 2023-09-04 06:01:28,050 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/info 2023-09-04 06:01:28,051 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 06:01:28,051 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:01:28,051 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 06:01:28,052 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/rep_barrier 2023-09-04 06:01:28,052 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/rep_barrier 2023-09-04 06:01:28,053 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 06:01:28,054 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:01:28,054 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 06:01:28,055 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/table 2023-09-04 06:01:28,055 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/table 2023-09-04 06:01:28,055 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 06:01:28,056 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:01:28,056 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740 2023-09-04 06:01:28,058 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740 2023-09-04 06:01:28,060 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 06:01:28,062 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 06:01:28,063 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=865785, jitterRate=0.10090306401252747}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 06:01:28,063 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 06:01:28,064 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693807288020 2023-09-04 06:01:28,068 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-09-04 06:01:28,068 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-09-04 06:01:28,069 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,44413,1693807287501, state=OPEN 2023-09-04 06:01:28,070 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-09-04 06:01:28,070 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 06:01:28,072 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-09-04 06:01:28,072 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,44413,1693807287501 in 204 msec 2023-09-04 06:01:28,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-09-04 06:01:28,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 363 msec 2023-09-04 06:01:28,076 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 415 msec 2023-09-04 06:01:28,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693807288076, completionTime=-1 2023-09-04 06:01:28,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-09-04 06:01:28,076 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-09-04 06:01:28,079 DEBUG [hconnection-0x5fca1d2e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 06:01:28,080 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48618, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 06:01:28,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-09-04 06:01:28,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693807348082 2023-09-04 06:01:28,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693807408082 2023-09-04 06:01:28,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-09-04 06:01:28,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46229,1693807287459-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:28,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46229,1693807287459-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:28,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46229,1693807287459-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:28,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:46229, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:28,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-09-04 06:01:28,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-09-04 06:01:28,089 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 06:01:28,090 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-09-04 06:01:28,090 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-09-04 06:01:28,091 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-09-04 06:01:28,092 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-04 06:01:28,094 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/.tmp/data/hbase/namespace/f33565f139fdc8b1ac198254ec83c339 2023-09-04 06:01:28,094 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/.tmp/data/hbase/namespace/f33565f139fdc8b1ac198254ec83c339 empty. 2023-09-04 06:01:28,095 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/.tmp/data/hbase/namespace/f33565f139fdc8b1ac198254ec83c339 2023-09-04 06:01:28,095 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-09-04 06:01:28,107 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-09-04 06:01:28,108 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => f33565f139fdc8b1ac198254ec83c339, NAME => 'hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/.tmp 2023-09-04 06:01:28,117 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:01:28,117 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing f33565f139fdc8b1ac198254ec83c339, disabling compactions & flushes 2023-09-04 06:01:28,117 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339. 2023-09-04 06:01:28,117 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339. 2023-09-04 06:01:28,117 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339. after waiting 0 ms 2023-09-04 06:01:28,117 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339. 2023-09-04 06:01:28,117 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339. 2023-09-04 06:01:28,117 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for f33565f139fdc8b1ac198254ec83c339: 2023-09-04 06:01:28,120 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-09-04 06:01:28,121 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693807288121"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693807288121"}]},"ts":"1693807288121"} 2023-09-04 06:01:28,124 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-04 06:01:28,125 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-04 06:01:28,125 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693807288125"}]},"ts":"1693807288125"} 2023-09-04 06:01:28,126 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-09-04 06:01:28,133 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f33565f139fdc8b1ac198254ec83c339, ASSIGN}] 2023-09-04 06:01:28,135 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f33565f139fdc8b1ac198254ec83c339, ASSIGN 2023-09-04 06:01:28,136 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=f33565f139fdc8b1ac198254ec83c339, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44413,1693807287501; forceNewPlan=false, retain=false 2023-09-04 06:01:28,287 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f33565f139fdc8b1ac198254ec83c339, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:01:28,288 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693807288287"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693807288287"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693807288287"}]},"ts":"1693807288287"} 2023-09-04 06:01:28,290 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure f33565f139fdc8b1ac198254ec83c339, server=jenkins-hbase4.apache.org,44413,1693807287501}] 2023-09-04 06:01:28,446 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339. 2023-09-04 06:01:28,447 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f33565f139fdc8b1ac198254ec83c339, NAME => 'hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339.', STARTKEY => '', ENDKEY => ''} 2023-09-04 06:01:28,447 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace f33565f139fdc8b1ac198254ec83c339 2023-09-04 06:01:28,447 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:01:28,447 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f33565f139fdc8b1ac198254ec83c339 2023-09-04 06:01:28,447 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f33565f139fdc8b1ac198254ec83c339 2023-09-04 06:01:28,448 INFO [StoreOpener-f33565f139fdc8b1ac198254ec83c339-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region f33565f139fdc8b1ac198254ec83c339 2023-09-04 06:01:28,450 DEBUG [StoreOpener-f33565f139fdc8b1ac198254ec83c339-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/namespace/f33565f139fdc8b1ac198254ec83c339/info 2023-09-04 06:01:28,450 DEBUG [StoreOpener-f33565f139fdc8b1ac198254ec83c339-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/namespace/f33565f139fdc8b1ac198254ec83c339/info 2023-09-04 06:01:28,450 INFO [StoreOpener-f33565f139fdc8b1ac198254ec83c339-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f33565f139fdc8b1ac198254ec83c339 columnFamilyName info 2023-09-04 06:01:28,451 INFO [StoreOpener-f33565f139fdc8b1ac198254ec83c339-1] regionserver.HStore(310): Store=f33565f139fdc8b1ac198254ec83c339/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:01:28,452 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/namespace/f33565f139fdc8b1ac198254ec83c339 2023-09-04 06:01:28,452 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/namespace/f33565f139fdc8b1ac198254ec83c339 2023-09-04 06:01:28,455 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f33565f139fdc8b1ac198254ec83c339 2023-09-04 06:01:28,457 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/namespace/f33565f139fdc8b1ac198254ec83c339/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 06:01:28,457 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f33565f139fdc8b1ac198254ec83c339; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=789150, jitterRate=0.0034567564725875854}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 06:01:28,458 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f33565f139fdc8b1ac198254ec83c339: 2023-09-04 06:01:28,458 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339., pid=6, masterSystemTime=1693807288442 2023-09-04 06:01:28,461 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339. 2023-09-04 06:01:28,461 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339. 2023-09-04 06:01:28,462 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f33565f139fdc8b1ac198254ec83c339, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:01:28,462 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693807288462"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693807288462"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693807288462"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693807288462"}]},"ts":"1693807288462"} 2023-09-04 06:01:28,466 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-09-04 06:01:28,466 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure f33565f139fdc8b1ac198254ec83c339, server=jenkins-hbase4.apache.org,44413,1693807287501 in 174 msec 2023-09-04 06:01:28,468 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-09-04 06:01:28,469 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=f33565f139fdc8b1ac198254ec83c339, ASSIGN in 333 msec 2023-09-04 06:01:28,470 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-04 06:01:28,471 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693807288470"}]},"ts":"1693807288470"} 2023-09-04 06:01:28,472 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-09-04 06:01:28,474 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-09-04 06:01:28,476 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 385 msec 2023-09-04 06:01:28,491 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-09-04 06:01:28,492 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-09-04 06:01:28,492 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:01:28,496 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-09-04 06:01:28,503 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 06:01:28,507 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 11 msec 2023-09-04 06:01:28,518 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-09-04 06:01:28,525 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 06:01:28,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 10 msec 2023-09-04 06:01:28,543 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-09-04 06:01:28,545 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-09-04 06:01:28,546 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.024sec 2023-09-04 06:01:28,546 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-09-04 06:01:28,546 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-09-04 06:01:28,546 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-09-04 06:01:28,546 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46229,1693807287459-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-09-04 06:01:28,546 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46229,1693807287459-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-09-04 06:01:28,548 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-09-04 06:01:28,615 DEBUG [Listener at localhost/37417] zookeeper.ReadOnlyZKClient(139): Connect 0x57b23b62 to 127.0.0.1:56951 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 06:01:28,620 DEBUG [Listener at localhost/37417] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@591785c5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 06:01:28,621 DEBUG [hconnection-0x22235f7f-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 06:01:28,623 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48626, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 06:01:28,627 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,46229,1693807287459 2023-09-04 06:01:28,627 INFO [Listener at localhost/37417] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:01:28,631 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-09-04 06:01:28,631 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:01:28,632 INFO [Listener at localhost/37417] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-09-04 06:01:28,634 DEBUG [Listener at localhost/37417] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-09-04 06:01:28,636 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42922, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-09-04 06:01:28,638 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46229] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-09-04 06:01:28,638 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46229] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-09-04 06:01:28,638 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46229] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-04 06:01:28,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46229] procedure2.ProcedureExecutor(1028): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRolling 2023-09-04 06:01:28,642 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-09-04 06:01:28,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46229] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRolling" procId is: 9 2023-09-04 06:01:28,644 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-04 06:01:28,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46229] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-04 06:01:28,646 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/.tmp/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92 2023-09-04 06:01:28,646 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/.tmp/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92 empty. 2023-09-04 06:01:28,647 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/.tmp/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92 2023-09-04 06:01:28,647 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRolling regions 2023-09-04 06:01:28,666 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/.tmp/data/default/TestLogRolling-testLogRolling/.tabledesc/.tableinfo.0000000001 2023-09-04 06:01:28,667 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => 999967069604f4f8061d4a09952cbc92, NAME => 'TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/.tmp 2023-09-04 06:01:28,677 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:01:28,677 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1604): Closing 999967069604f4f8061d4a09952cbc92, disabling compactions & flushes 2023-09-04 06:01:28,677 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. 2023-09-04 06:01:28,677 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. 2023-09-04 06:01:28,677 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. after waiting 0 ms 2023-09-04 06:01:28,677 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. 2023-09-04 06:01:28,677 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. 2023-09-04 06:01:28,677 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1558): Region close journal for 999967069604f4f8061d4a09952cbc92: 2023-09-04 06:01:28,681 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ADD_TO_META 2023-09-04 06:01:28,683 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693807288682"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693807288682"}]},"ts":"1693807288682"} 2023-09-04 06:01:28,685 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-04 06:01:28,686 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-04 06:01:28,686 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693807288686"}]},"ts":"1693807288686"} 2023-09-04 06:01:28,688 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLING in hbase:meta 2023-09-04 06:01:28,691 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=999967069604f4f8061d4a09952cbc92, ASSIGN}] 2023-09-04 06:01:28,693 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=999967069604f4f8061d4a09952cbc92, ASSIGN 2023-09-04 06:01:28,694 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=999967069604f4f8061d4a09952cbc92, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44413,1693807287501; forceNewPlan=false, retain=false 2023-09-04 06:01:28,845 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=999967069604f4f8061d4a09952cbc92, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:01:28,845 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693807288845"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693807288845"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693807288845"}]},"ts":"1693807288845"} 2023-09-04 06:01:28,847 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 999967069604f4f8061d4a09952cbc92, server=jenkins-hbase4.apache.org,44413,1693807287501}] 2023-09-04 06:01:29,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. 2023-09-04 06:01:29,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 999967069604f4f8061d4a09952cbc92, NAME => 'TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.', STARTKEY => '', ENDKEY => ''} 2023-09-04 06:01:29,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling 999967069604f4f8061d4a09952cbc92 2023-09-04 06:01:29,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:01:29,005 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 999967069604f4f8061d4a09952cbc92 2023-09-04 06:01:29,005 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 999967069604f4f8061d4a09952cbc92 2023-09-04 06:01:29,006 INFO [StoreOpener-999967069604f4f8061d4a09952cbc92-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 999967069604f4f8061d4a09952cbc92 2023-09-04 06:01:29,007 DEBUG [StoreOpener-999967069604f4f8061d4a09952cbc92-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info 2023-09-04 06:01:29,007 DEBUG [StoreOpener-999967069604f4f8061d4a09952cbc92-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info 2023-09-04 06:01:29,008 INFO [StoreOpener-999967069604f4f8061d4a09952cbc92-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 999967069604f4f8061d4a09952cbc92 columnFamilyName info 2023-09-04 06:01:29,008 INFO [StoreOpener-999967069604f4f8061d4a09952cbc92-1] regionserver.HStore(310): Store=999967069604f4f8061d4a09952cbc92/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:01:29,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92 2023-09-04 06:01:29,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92 2023-09-04 06:01:29,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 999967069604f4f8061d4a09952cbc92 2023-09-04 06:01:29,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 06:01:29,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 999967069604f4f8061d4a09952cbc92; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=770330, jitterRate=-0.02047482132911682}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 06:01:29,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 999967069604f4f8061d4a09952cbc92: 2023-09-04 06:01:29,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92., pid=11, masterSystemTime=1693807289000 2023-09-04 06:01:29,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. 2023-09-04 06:01:29,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. 2023-09-04 06:01:29,019 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=999967069604f4f8061d4a09952cbc92, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:01:29,019 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693807289019"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693807289019"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693807289019"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693807289019"}]},"ts":"1693807289019"} 2023-09-04 06:01:29,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=11, resume processing ppid=10 2023-09-04 06:01:29,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 999967069604f4f8061d4a09952cbc92, server=jenkins-hbase4.apache.org,44413,1693807287501 in 174 msec 2023-09-04 06:01:29,026 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=10, resume processing ppid=9 2023-09-04 06:01:29,026 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=999967069604f4f8061d4a09952cbc92, ASSIGN in 332 msec 2023-09-04 06:01:29,027 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-04 06:01:29,027 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693807289027"}]},"ts":"1693807289027"} 2023-09-04 06:01:29,029 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLED in hbase:meta 2023-09-04 06:01:29,032 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_POST_OPERATION 2023-09-04 06:01:29,034 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRolling in 394 msec 2023-09-04 06:01:31,724 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-04 06:01:33,745 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-09-04 06:01:33,746 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-09-04 06:01:33,746 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRolling' 2023-09-04 06:01:38,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46229] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-04 06:01:38,646 INFO [Listener at localhost/37417] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRolling, procId: 9 completed 2023-09-04 06:01:38,648 DEBUG [Listener at localhost/37417] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRolling 2023-09-04 06:01:38,648 DEBUG [Listener at localhost/37417] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. 2023-09-04 06:01:38,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on 999967069604f4f8061d4a09952cbc92 2023-09-04 06:01:38,664 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 999967069604f4f8061d4a09952cbc92 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 06:01:38,681 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=12 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp/info/6e01398552aa463d8e6b1aa429c58b80 2023-09-04 06:01:38,691 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp/info/6e01398552aa463d8e6b1aa429c58b80 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/6e01398552aa463d8e6b1aa429c58b80 2023-09-04 06:01:38,697 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/6e01398552aa463d8e6b1aa429c58b80, entries=8, sequenceid=12, filesize=13.0 K 2023-09-04 06:01:38,697 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=999967069604f4f8061d4a09952cbc92, server=jenkins-hbase4.apache.org,44413,1693807287501 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 06:01:38,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] ipc.CallRunner(144): callId: 38 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:48626 deadline: 1693807308697, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=999967069604f4f8061d4a09952cbc92, server=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:01:38,699 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 999967069604f4f8061d4a09952cbc92 in 35ms, sequenceid=12, compaction requested=false 2023-09-04 06:01:38,699 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 999967069604f4f8061d4a09952cbc92: 2023-09-04 06:01:48,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on 999967069604f4f8061d4a09952cbc92 2023-09-04 06:01:48,773 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 999967069604f4f8061d4a09952cbc92 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-04 06:01:48,791 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=37 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp/info/a19ce576bbb349be9f8251d5954dae19 2023-09-04 06:01:48,798 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp/info/a19ce576bbb349be9f8251d5954dae19 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/a19ce576bbb349be9f8251d5954dae19 2023-09-04 06:01:48,804 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/a19ce576bbb349be9f8251d5954dae19, entries=22, sequenceid=37, filesize=27.4 K 2023-09-04 06:01:48,805 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=2.05 KB/2102 for 999967069604f4f8061d4a09952cbc92 in 32ms, sequenceid=37, compaction requested=false 2023-09-04 06:01:48,805 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 999967069604f4f8061d4a09952cbc92: 2023-09-04 06:01:48,806 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=40.3 K, sizeToCheck=16.0 K 2023-09-04 06:01:48,806 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-04 06:01:48,807 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/a19ce576bbb349be9f8251d5954dae19 because midkey is the same as first or last row 2023-09-04 06:01:50,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on 999967069604f4f8061d4a09952cbc92 2023-09-04 06:01:50,786 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 999967069604f4f8061d4a09952cbc92 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 06:01:50,806 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=48 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp/info/755207f809904b7ab301ac7102384122 2023-09-04 06:01:50,816 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp/info/755207f809904b7ab301ac7102384122 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/755207f809904b7ab301ac7102384122 2023-09-04 06:01:50,824 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=999967069604f4f8061d4a09952cbc92, server=jenkins-hbase4.apache.org,44413,1693807287501 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 06:01:50,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] ipc.CallRunner(144): callId: 70 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:48626 deadline: 1693807320824, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=999967069604f4f8061d4a09952cbc92, server=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:01:50,827 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/755207f809904b7ab301ac7102384122, entries=8, sequenceid=48, filesize=13.0 K 2023-09-04 06:01:50,829 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 999967069604f4f8061d4a09952cbc92 in 43ms, sequenceid=48, compaction requested=true 2023-09-04 06:01:50,829 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 999967069604f4f8061d4a09952cbc92: 2023-09-04 06:01:50,829 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=53.3 K, sizeToCheck=16.0 K 2023-09-04 06:01:50,829 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-04 06:01:50,829 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/a19ce576bbb349be9f8251d5954dae19 because midkey is the same as first or last row 2023-09-04 06:01:50,831 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:01:50,831 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 06:01:50,834 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 54576 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 06:01:50,834 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1912): 999967069604f4f8061d4a09952cbc92/info is initiating minor compaction (all files) 2023-09-04 06:01:50,834 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 999967069604f4f8061d4a09952cbc92/info in TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. 2023-09-04 06:01:50,834 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/6e01398552aa463d8e6b1aa429c58b80, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/a19ce576bbb349be9f8251d5954dae19, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/755207f809904b7ab301ac7102384122] into tmpdir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp, totalSize=53.3 K 2023-09-04 06:01:50,835 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting 6e01398552aa463d8e6b1aa429c58b80, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1693807298651 2023-09-04 06:01:50,835 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting a19ce576bbb349be9f8251d5954dae19, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=37, earliestPutTs=1693807298665 2023-09-04 06:01:50,836 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting 755207f809904b7ab301ac7102384122, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1693807308774 2023-09-04 06:01:50,854 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] throttle.PressureAwareThroughputController(145): 999967069604f4f8061d4a09952cbc92#info#compaction#18 average throughput is 19.04 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 06:01:50,876 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp/info/797d9a64319545349a34d7817a42ef48 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/797d9a64319545349a34d7817a42ef48 2023-09-04 06:01:50,882 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 999967069604f4f8061d4a09952cbc92/info of 999967069604f4f8061d4a09952cbc92 into 797d9a64319545349a34d7817a42ef48(size=43.9 K), total size for store is 43.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 06:01:50,882 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 999967069604f4f8061d4a09952cbc92: 2023-09-04 06:01:50,882 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92., storeName=999967069604f4f8061d4a09952cbc92/info, priority=13, startTime=1693807310831; duration=0sec 2023-09-04 06:01:50,884 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=43.9 K, sizeToCheck=16.0 K 2023-09-04 06:01:50,884 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-04 06:01:50,884 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.StoreUtils(129): cannot split hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/797d9a64319545349a34d7817a42ef48 because midkey is the same as first or last row 2023-09-04 06:01:50,884 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:02:00,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on 999967069604f4f8061d4a09952cbc92 2023-09-04 06:02:00,851 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 999967069604f4f8061d4a09952cbc92 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-04 06:02:00,865 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=74 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp/info/df632870c4e142a2b8d7f9d2faf76b6e 2023-09-04 06:02:00,872 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp/info/df632870c4e142a2b8d7f9d2faf76b6e as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/df632870c4e142a2b8d7f9d2faf76b6e 2023-09-04 06:02:00,878 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/df632870c4e142a2b8d7f9d2faf76b6e, entries=22, sequenceid=74, filesize=27.4 K 2023-09-04 06:02:00,879 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=4.11 KB/4204 for 999967069604f4f8061d4a09952cbc92 in 28ms, sequenceid=74, compaction requested=false 2023-09-04 06:02:00,879 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 999967069604f4f8061d4a09952cbc92: 2023-09-04 06:02:00,879 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=71.3 K, sizeToCheck=16.0 K 2023-09-04 06:02:00,879 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-04 06:02:00,879 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/797d9a64319545349a34d7817a42ef48 because midkey is the same as first or last row 2023-09-04 06:02:02,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on 999967069604f4f8061d4a09952cbc92 2023-09-04 06:02:02,863 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 999967069604f4f8061d4a09952cbc92 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 06:02:02,877 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=85 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp/info/bfc2aed4f0f747228a6930eec2e50488 2023-09-04 06:02:02,884 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp/info/bfc2aed4f0f747228a6930eec2e50488 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/bfc2aed4f0f747228a6930eec2e50488 2023-09-04 06:02:02,890 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/bfc2aed4f0f747228a6930eec2e50488, entries=8, sequenceid=85, filesize=13.0 K 2023-09-04 06:02:02,890 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=999967069604f4f8061d4a09952cbc92, server=jenkins-hbase4.apache.org,44413,1693807287501 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 06:02:02,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] ipc.CallRunner(144): callId: 102 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:48626 deadline: 1693807332890, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=999967069604f4f8061d4a09952cbc92, server=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:02:02,891 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 999967069604f4f8061d4a09952cbc92 in 28ms, sequenceid=85, compaction requested=true 2023-09-04 06:02:02,891 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 999967069604f4f8061d4a09952cbc92: 2023-09-04 06:02:02,891 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=84.3 K, sizeToCheck=16.0 K 2023-09-04 06:02:02,891 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-04 06:02:02,891 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/797d9a64319545349a34d7817a42ef48 because midkey is the same as first or last row 2023-09-04 06:02:02,891 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:02:02,891 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 06:02:02,892 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 86295 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 06:02:02,893 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1912): 999967069604f4f8061d4a09952cbc92/info is initiating minor compaction (all files) 2023-09-04 06:02:02,893 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 999967069604f4f8061d4a09952cbc92/info in TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. 2023-09-04 06:02:02,893 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/797d9a64319545349a34d7817a42ef48, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/df632870c4e142a2b8d7f9d2faf76b6e, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/bfc2aed4f0f747228a6930eec2e50488] into tmpdir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp, totalSize=84.3 K 2023-09-04 06:02:02,893 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting 797d9a64319545349a34d7817a42ef48, keycount=38, bloomtype=ROW, size=43.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1693807298651 2023-09-04 06:02:02,894 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting df632870c4e142a2b8d7f9d2faf76b6e, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=74, earliestPutTs=1693807310786 2023-09-04 06:02:02,894 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting bfc2aed4f0f747228a6930eec2e50488, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1693807320852 2023-09-04 06:02:02,907 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] throttle.PressureAwareThroughputController(145): 999967069604f4f8061d4a09952cbc92#info#compaction#21 average throughput is 17.04 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 06:02:02,922 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp/info/e7a41f39b10b4f1382ba5803301ef10a as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/e7a41f39b10b4f1382ba5803301ef10a 2023-09-04 06:02:02,928 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 999967069604f4f8061d4a09952cbc92/info of 999967069604f4f8061d4a09952cbc92 into e7a41f39b10b4f1382ba5803301ef10a(size=75.0 K), total size for store is 75.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 06:02:02,928 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 999967069604f4f8061d4a09952cbc92: 2023-09-04 06:02:02,928 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92., storeName=999967069604f4f8061d4a09952cbc92/info, priority=13, startTime=1693807322891; duration=0sec 2023-09-04 06:02:02,929 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=75.0 K, sizeToCheck=16.0 K 2023-09-04 06:02:02,929 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-04 06:02:02,930 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit(227): Splitting TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92., compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:02:02,930 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:02:02,931 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=46229] assignment.AssignmentManager(1140): Split request from jenkins-hbase4.apache.org,44413,1693807287501, parent={ENCODED => 999967069604f4f8061d4a09952cbc92, NAME => 'TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.', STARTKEY => '', ENDKEY => ''} splitKey=row0064 2023-09-04 06:02:02,938 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=46229] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:02:02,944 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=46229] procedure2.ProcedureExecutor(1028): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=999967069604f4f8061d4a09952cbc92, daughterA=c2695740c88d7b4dd012810de5537468, daughterB=c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:02:02,945 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=999967069604f4f8061d4a09952cbc92, daughterA=c2695740c88d7b4dd012810de5537468, daughterB=c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:02:02,945 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=999967069604f4f8061d4a09952cbc92, daughterA=c2695740c88d7b4dd012810de5537468, daughterB=c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:02:02,945 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=999967069604f4f8061d4a09952cbc92, daughterA=c2695740c88d7b4dd012810de5537468, daughterB=c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:02:02,953 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=999967069604f4f8061d4a09952cbc92, UNASSIGN}] 2023-09-04 06:02:02,954 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=999967069604f4f8061d4a09952cbc92, UNASSIGN 2023-09-04 06:02:02,955 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=999967069604f4f8061d4a09952cbc92, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:02:02,955 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693807322955"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693807322955"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693807322955"}]},"ts":"1693807322955"} 2023-09-04 06:02:02,957 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 999967069604f4f8061d4a09952cbc92, server=jenkins-hbase4.apache.org,44413,1693807287501}] 2023-09-04 06:02:03,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 999967069604f4f8061d4a09952cbc92 2023-09-04 06:02:03,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 999967069604f4f8061d4a09952cbc92, disabling compactions & flushes 2023-09-04 06:02:03,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. 2023-09-04 06:02:03,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. 2023-09-04 06:02:03,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. after waiting 0 ms 2023-09-04 06:02:03,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. 2023-09-04 06:02:03,116 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 999967069604f4f8061d4a09952cbc92 1/1 column families, dataSize=21.55 KB heapSize=23.38 KB 2023-09-04 06:02:03,127 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=21.55 KB at sequenceid=110 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp/info/5cacbf7fe3864061b8a9e84eeab731ed 2023-09-04 06:02:03,133 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.tmp/info/5cacbf7fe3864061b8a9e84eeab731ed as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/5cacbf7fe3864061b8a9e84eeab731ed 2023-09-04 06:02:03,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/5cacbf7fe3864061b8a9e84eeab731ed, entries=21, sequenceid=110, filesize=26.3 K 2023-09-04 06:02:03,140 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~21.55 KB/22071, heapSize ~23.37 KB/23928, currentSize=0 B/0 for 999967069604f4f8061d4a09952cbc92 in 25ms, sequenceid=110, compaction requested=false 2023-09-04 06:02:03,146 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/6e01398552aa463d8e6b1aa429c58b80, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/a19ce576bbb349be9f8251d5954dae19, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/797d9a64319545349a34d7817a42ef48, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/755207f809904b7ab301ac7102384122, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/df632870c4e142a2b8d7f9d2faf76b6e, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/bfc2aed4f0f747228a6930eec2e50488] to archive 2023-09-04 06:02:03,147 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-09-04 06:02:03,149 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/6e01398552aa463d8e6b1aa429c58b80 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/6e01398552aa463d8e6b1aa429c58b80 2023-09-04 06:02:03,150 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/a19ce576bbb349be9f8251d5954dae19 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/a19ce576bbb349be9f8251d5954dae19 2023-09-04 06:02:03,152 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/797d9a64319545349a34d7817a42ef48 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/797d9a64319545349a34d7817a42ef48 2023-09-04 06:02:03,154 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/755207f809904b7ab301ac7102384122 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/755207f809904b7ab301ac7102384122 2023-09-04 06:02:03,155 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/df632870c4e142a2b8d7f9d2faf76b6e to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/df632870c4e142a2b8d7f9d2faf76b6e 2023-09-04 06:02:03,156 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/bfc2aed4f0f747228a6930eec2e50488 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/bfc2aed4f0f747228a6930eec2e50488 2023-09-04 06:02:03,163 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=1 2023-09-04 06:02:03,164 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. 2023-09-04 06:02:03,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 999967069604f4f8061d4a09952cbc92: 2023-09-04 06:02:03,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 999967069604f4f8061d4a09952cbc92 2023-09-04 06:02:03,166 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=999967069604f4f8061d4a09952cbc92, regionState=CLOSED 2023-09-04 06:02:03,167 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693807323166"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693807323166"}]},"ts":"1693807323166"} 2023-09-04 06:02:03,170 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=14, resume processing ppid=13 2023-09-04 06:02:03,170 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 999967069604f4f8061d4a09952cbc92, server=jenkins-hbase4.apache.org,44413,1693807287501 in 211 msec 2023-09-04 06:02:03,172 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=13, resume processing ppid=12 2023-09-04 06:02:03,172 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=999967069604f4f8061d4a09952cbc92, UNASSIGN in 217 msec 2023-09-04 06:02:03,183 INFO [PEWorker-4] assignment.SplitTableRegionProcedure(694): pid=12 splitting 2 storefiles, region=999967069604f4f8061d4a09952cbc92, threads=2 2023-09-04 06:02:03,184 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/5cacbf7fe3864061b8a9e84eeab731ed for region: 999967069604f4f8061d4a09952cbc92 2023-09-04 06:02:03,184 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/e7a41f39b10b4f1382ba5803301ef10a for region: 999967069604f4f8061d4a09952cbc92 2023-09-04 06:02:03,194 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/5cacbf7fe3864061b8a9e84eeab731ed, top=true 2023-09-04 06:02:03,204 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/.splits/c729f1dd7f7ce1a44f1822ce19e5bd40/info/TestLogRolling-testLogRolling=999967069604f4f8061d4a09952cbc92-5cacbf7fe3864061b8a9e84eeab731ed for child: c729f1dd7f7ce1a44f1822ce19e5bd40, parent: 999967069604f4f8061d4a09952cbc92 2023-09-04 06:02:03,205 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/5cacbf7fe3864061b8a9e84eeab731ed for region: 999967069604f4f8061d4a09952cbc92 2023-09-04 06:02:03,219 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/e7a41f39b10b4f1382ba5803301ef10a for region: 999967069604f4f8061d4a09952cbc92 2023-09-04 06:02:03,219 DEBUG [PEWorker-4] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 999967069604f4f8061d4a09952cbc92 Daughter A: 1 storefiles, Daughter B: 2 storefiles. 2023-09-04 06:02:03,246 DEBUG [PEWorker-4] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c2695740c88d7b4dd012810de5537468/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=-1 2023-09-04 06:02:03,248 DEBUG [PEWorker-4] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=-1 2023-09-04 06:02:03,250 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693807323250"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1693807323250"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1693807323250"}]},"ts":"1693807323250"} 2023-09-04 06:02:03,250 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693807323250"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693807323250"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693807323250"}]},"ts":"1693807323250"} 2023-09-04 06:02:03,251 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693807323250"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693807323250"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693807323250"}]},"ts":"1693807323250"} 2023-09-04 06:02:03,291 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44413] regionserver.HRegion(9158): Flush requested on 1588230740 2023-09-04 06:02:03,291 DEBUG [MemStoreFlusher.0] regionserver.FlushAllLargeStoresPolicy(69): Since none of the CFs were above the size, flushing all. 2023-09-04 06:02:03,291 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=4.82 KB heapSize=8.36 KB 2023-09-04 06:02:03,300 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=c2695740c88d7b4dd012810de5537468, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=c729f1dd7f7ce1a44f1822ce19e5bd40, ASSIGN}] 2023-09-04 06:02:03,302 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=c2695740c88d7b4dd012810de5537468, ASSIGN 2023-09-04 06:02:03,302 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=c729f1dd7f7ce1a44f1822ce19e5bd40, ASSIGN 2023-09-04 06:02:03,305 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=c2695740c88d7b4dd012810de5537468, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44413,1693807287501; forceNewPlan=false, retain=false 2023-09-04 06:02:03,305 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=c729f1dd7f7ce1a44f1822ce19e5bd40, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44413,1693807287501; forceNewPlan=false, retain=false 2023-09-04 06:02:03,306 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.61 KB at sequenceid=17 (bloomFilter=false), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/.tmp/info/344ee00b5d9c4dbf89b697df0a9e5448 2023-09-04 06:02:03,321 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=216 B at sequenceid=17 (bloomFilter=false), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/.tmp/table/62d781a47aab4dc69f0b800f4b77c7bd 2023-09-04 06:02:03,327 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/.tmp/info/344ee00b5d9c4dbf89b697df0a9e5448 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/info/344ee00b5d9c4dbf89b697df0a9e5448 2023-09-04 06:02:03,333 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/info/344ee00b5d9c4dbf89b697df0a9e5448, entries=29, sequenceid=17, filesize=8.6 K 2023-09-04 06:02:03,334 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/.tmp/table/62d781a47aab4dc69f0b800f4b77c7bd as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/table/62d781a47aab4dc69f0b800f4b77c7bd 2023-09-04 06:02:03,340 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/table/62d781a47aab4dc69f0b800f4b77c7bd, entries=4, sequenceid=17, filesize=4.8 K 2023-09-04 06:02:03,341 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~4.82 KB/4934, heapSize ~8.08 KB/8272, currentSize=0 B/0 for 1588230740 in 50ms, sequenceid=17, compaction requested=false 2023-09-04 06:02:03,342 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-09-04 06:02:03,456 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=c2695740c88d7b4dd012810de5537468, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:02:03,456 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=c729f1dd7f7ce1a44f1822ce19e5bd40, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:02:03,457 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693807323456"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693807323456"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693807323456"}]},"ts":"1693807323456"} 2023-09-04 06:02:03,457 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693807323456"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693807323456"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693807323456"}]},"ts":"1693807323456"} 2023-09-04 06:02:03,459 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure c2695740c88d7b4dd012810de5537468, server=jenkins-hbase4.apache.org,44413,1693807287501}] 2023-09-04 06:02:03,459 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501}] 2023-09-04 06:02:03,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468. 2023-09-04 06:02:03,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c2695740c88d7b4dd012810de5537468, NAME => 'TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468.', STARTKEY => '', ENDKEY => 'row0064'} 2023-09-04 06:02:03,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling c2695740c88d7b4dd012810de5537468 2023-09-04 06:02:03,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:02:03,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c2695740c88d7b4dd012810de5537468 2023-09-04 06:02:03,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c2695740c88d7b4dd012810de5537468 2023-09-04 06:02:03,617 INFO [StoreOpener-c2695740c88d7b4dd012810de5537468-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region c2695740c88d7b4dd012810de5537468 2023-09-04 06:02:03,618 DEBUG [StoreOpener-c2695740c88d7b4dd012810de5537468-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c2695740c88d7b4dd012810de5537468/info 2023-09-04 06:02:03,618 DEBUG [StoreOpener-c2695740c88d7b4dd012810de5537468-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c2695740c88d7b4dd012810de5537468/info 2023-09-04 06:02:03,618 INFO [StoreOpener-c2695740c88d7b4dd012810de5537468-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c2695740c88d7b4dd012810de5537468 columnFamilyName info 2023-09-04 06:02:03,632 DEBUG [StoreOpener-c2695740c88d7b4dd012810de5537468-1] regionserver.HStore(539): loaded hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c2695740c88d7b4dd012810de5537468/info/e7a41f39b10b4f1382ba5803301ef10a.999967069604f4f8061d4a09952cbc92->hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/e7a41f39b10b4f1382ba5803301ef10a-bottom 2023-09-04 06:02:03,633 INFO [StoreOpener-c2695740c88d7b4dd012810de5537468-1] regionserver.HStore(310): Store=c2695740c88d7b4dd012810de5537468/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:02:03,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c2695740c88d7b4dd012810de5537468 2023-09-04 06:02:03,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c2695740c88d7b4dd012810de5537468 2023-09-04 06:02:03,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c2695740c88d7b4dd012810de5537468 2023-09-04 06:02:03,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c2695740c88d7b4dd012810de5537468; next sequenceid=114; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=814921, jitterRate=0.03622664511203766}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 06:02:03,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c2695740c88d7b4dd012810de5537468: 2023-09-04 06:02:03,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468., pid=17, masterSystemTime=1693807323611 2023-09-04 06:02:03,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:02:03,642 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-09-04 06:02:03,643 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468. 2023-09-04 06:02:03,643 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1912): c2695740c88d7b4dd012810de5537468/info is initiating minor compaction (all files) 2023-09-04 06:02:03,643 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c2695740c88d7b4dd012810de5537468/info in TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468. 2023-09-04 06:02:03,643 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c2695740c88d7b4dd012810de5537468/info/e7a41f39b10b4f1382ba5803301ef10a.999967069604f4f8061d4a09952cbc92->hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/e7a41f39b10b4f1382ba5803301ef10a-bottom] into tmpdir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c2695740c88d7b4dd012810de5537468/.tmp, totalSize=75.0 K 2023-09-04 06:02:03,644 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting e7a41f39b10b4f1382ba5803301ef10a.999967069604f4f8061d4a09952cbc92, keycount=34, bloomtype=ROW, size=75.0 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1693807298651 2023-09-04 06:02:03,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468. 2023-09-04 06:02:03,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468. 2023-09-04 06:02:03,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:02:03,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c729f1dd7f7ce1a44f1822ce19e5bd40, NAME => 'TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40.', STARTKEY => 'row0064', ENDKEY => ''} 2023-09-04 06:02:03,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:02:03,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:02:03,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:02:03,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:02:03,646 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=c2695740c88d7b4dd012810de5537468, regionState=OPEN, openSeqNum=114, regionLocation=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:02:03,646 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693807323646"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693807323646"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693807323646"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693807323646"}]},"ts":"1693807323646"} 2023-09-04 06:02:03,647 INFO [StoreOpener-c729f1dd7f7ce1a44f1822ce19e5bd40-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:02:03,648 DEBUG [StoreOpener-c729f1dd7f7ce1a44f1822ce19e5bd40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info 2023-09-04 06:02:03,648 DEBUG [StoreOpener-c729f1dd7f7ce1a44f1822ce19e5bd40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info 2023-09-04 06:02:03,649 INFO [StoreOpener-c729f1dd7f7ce1a44f1822ce19e5bd40-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c729f1dd7f7ce1a44f1822ce19e5bd40 columnFamilyName info 2023-09-04 06:02:03,651 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=17, resume processing ppid=15 2023-09-04 06:02:03,652 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure c2695740c88d7b4dd012810de5537468, server=jenkins-hbase4.apache.org,44413,1693807287501 in 190 msec 2023-09-04 06:02:03,654 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=c2695740c88d7b4dd012810de5537468, ASSIGN in 352 msec 2023-09-04 06:02:03,654 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] throttle.PressureAwareThroughputController(145): c2695740c88d7b4dd012810de5537468#info#compaction#25 average throughput is 15.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 06:02:03,665 DEBUG [StoreOpener-c729f1dd7f7ce1a44f1822ce19e5bd40-1] regionserver.HStore(539): loaded hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/TestLogRolling-testLogRolling=999967069604f4f8061d4a09952cbc92-5cacbf7fe3864061b8a9e84eeab731ed 2023-09-04 06:02:03,672 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c2695740c88d7b4dd012810de5537468/.tmp/info/fe97ed24bc124c218522bfddc03ffafd as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c2695740c88d7b4dd012810de5537468/info/fe97ed24bc124c218522bfddc03ffafd 2023-09-04 06:02:03,672 DEBUG [StoreOpener-c729f1dd7f7ce1a44f1822ce19e5bd40-1] regionserver.HStore(539): loaded hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e7a41f39b10b4f1382ba5803301ef10a.999967069604f4f8061d4a09952cbc92->hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/e7a41f39b10b4f1382ba5803301ef10a-top 2023-09-04 06:02:03,673 INFO [StoreOpener-c729f1dd7f7ce1a44f1822ce19e5bd40-1] regionserver.HStore(310): Store=c729f1dd7f7ce1a44f1822ce19e5bd40/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:02:03,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:02:03,675 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:02:03,679 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:02:03,680 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 1 (all) file(s) in c2695740c88d7b4dd012810de5537468/info of c2695740c88d7b4dd012810de5537468 into fe97ed24bc124c218522bfddc03ffafd(size=69.7 K), total size for store is 69.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 06:02:03,680 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c2695740c88d7b4dd012810de5537468: 2023-09-04 06:02:03,680 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468., storeName=c2695740c88d7b4dd012810de5537468/info, priority=15, startTime=1693807323641; duration=0sec 2023-09-04 06:02:03,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c729f1dd7f7ce1a44f1822ce19e5bd40; next sequenceid=114; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=746198, jitterRate=-0.05116039514541626}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 06:02:03,680 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:02:03,680 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:02:03,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40., pid=18, masterSystemTime=1693807323611 2023-09-04 06:02:03,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:02:03,682 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-09-04 06:02:03,683 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:02:03,683 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1912): c729f1dd7f7ce1a44f1822ce19e5bd40/info is initiating minor compaction (all files) 2023-09-04 06:02:03,683 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c729f1dd7f7ce1a44f1822ce19e5bd40/info in TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:02:03,683 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e7a41f39b10b4f1382ba5803301ef10a.999967069604f4f8061d4a09952cbc92->hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/e7a41f39b10b4f1382ba5803301ef10a-top, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/TestLogRolling-testLogRolling=999967069604f4f8061d4a09952cbc92-5cacbf7fe3864061b8a9e84eeab731ed] into tmpdir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp, totalSize=101.3 K 2023-09-04 06:02:03,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:02:03,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:02:03,684 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting e7a41f39b10b4f1382ba5803301ef10a.999967069604f4f8061d4a09952cbc92, keycount=34, bloomtype=ROW, size=75.0 K, encoding=NONE, compression=NONE, seqNum=86, earliestPutTs=1693807298651 2023-09-04 06:02:03,685 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=c729f1dd7f7ce1a44f1822ce19e5bd40, regionState=OPEN, openSeqNum=114, regionLocation=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:02:03,685 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting TestLogRolling-testLogRolling=999967069604f4f8061d4a09952cbc92-5cacbf7fe3864061b8a9e84eeab731ed, keycount=21, bloomtype=ROW, size=26.3 K, encoding=NONE, compression=NONE, seqNum=110, earliestPutTs=1693807322864 2023-09-04 06:02:03,685 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693807323685"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693807323685"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693807323685"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693807323685"}]},"ts":"1693807323685"} 2023-09-04 06:02:03,689 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=18, resume processing ppid=16 2023-09-04 06:02:03,690 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501 in 228 msec 2023-09-04 06:02:03,692 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=16, resume processing ppid=12 2023-09-04 06:02:03,693 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=c729f1dd7f7ce1a44f1822ce19e5bd40, ASSIGN in 390 msec 2023-09-04 06:02:03,695 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=999967069604f4f8061d4a09952cbc92, daughterA=c2695740c88d7b4dd012810de5537468, daughterB=c729f1dd7f7ce1a44f1822ce19e5bd40 in 755 msec 2023-09-04 06:02:03,696 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] throttle.PressureAwareThroughputController(145): c729f1dd7f7ce1a44f1822ce19e5bd40#info#compaction#26 average throughput is 26.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 06:02:03,713 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/e96f84b2c0f84929a0e6315a70d5ca18 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e96f84b2c0f84929a0e6315a70d5ca18 2023-09-04 06:02:03,720 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 (all) file(s) in c729f1dd7f7ce1a44f1822ce19e5bd40/info of c729f1dd7f7ce1a44f1822ce19e5bd40 into e96f84b2c0f84929a0e6315a70d5ca18(size=31.6 K), total size for store is 31.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 06:02:03,721 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:02:03,721 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40., storeName=c729f1dd7f7ce1a44f1822ce19e5bd40/info, priority=14, startTime=1693807323681; duration=0sec 2023-09-04 06:02:03,721 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:02:08,706 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-04 06:02:12,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] ipc.CallRunner(144): callId: 104 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:48626 deadline: 1693807342913, exception=org.apache.hadoop.hbase.NotServingRegionException: TestLogRolling-testLogRolling,,1693807288638.999967069604f4f8061d4a09952cbc92. is not online on jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:02:35,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:02:35,111 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing c729f1dd7f7ce1a44f1822ce19e5bd40 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 06:02:35,131 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=125 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/e73daf21f30342d79a91e798dcb6403b 2023-09-04 06:02:35,137 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/e73daf21f30342d79a91e798dcb6403b as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e73daf21f30342d79a91e798dcb6403b 2023-09-04 06:02:35,141 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 06:02:35,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] ipc.CallRunner(144): callId: 137 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:48626 deadline: 1693807365140, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:02:35,144 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e73daf21f30342d79a91e798dcb6403b, entries=8, sequenceid=125, filesize=13.0 K 2023-09-04 06:02:35,145 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for c729f1dd7f7ce1a44f1822ce19e5bd40 in 34ms, sequenceid=125, compaction requested=false 2023-09-04 06:02:35,145 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:02:45,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:02:45,199 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing c729f1dd7f7ce1a44f1822ce19e5bd40 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-04 06:02:45,210 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 06:02:45,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] ipc.CallRunner(144): callId: 147 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:48626 deadline: 1693807375210, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:02:45,215 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=150 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/4a5fd927b9664c57bdcf0ebbbe62693c 2023-09-04 06:02:45,221 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/4a5fd927b9664c57bdcf0ebbbe62693c as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/4a5fd927b9664c57bdcf0ebbbe62693c 2023-09-04 06:02:45,227 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/4a5fd927b9664c57bdcf0ebbbe62693c, entries=22, sequenceid=150, filesize=27.4 K 2023-09-04 06:02:45,228 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=7.18 KB/7357 for c729f1dd7f7ce1a44f1822ce19e5bd40 in 29ms, sequenceid=150, compaction requested=true 2023-09-04 06:02:45,228 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:02:45,228 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:02:45,228 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 06:02:45,229 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 73719 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 06:02:45,229 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1912): c729f1dd7f7ce1a44f1822ce19e5bd40/info is initiating minor compaction (all files) 2023-09-04 06:02:45,229 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c729f1dd7f7ce1a44f1822ce19e5bd40/info in TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:02:45,230 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e96f84b2c0f84929a0e6315a70d5ca18, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e73daf21f30342d79a91e798dcb6403b, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/4a5fd927b9664c57bdcf0ebbbe62693c] into tmpdir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp, totalSize=72.0 K 2023-09-04 06:02:45,230 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting e96f84b2c0f84929a0e6315a70d5ca18, keycount=26, bloomtype=ROW, size=31.6 K, encoding=NONE, compression=NONE, seqNum=110, earliestPutTs=1693807320856 2023-09-04 06:02:45,231 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting e73daf21f30342d79a91e798dcb6403b, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=125, earliestPutTs=1693807353102 2023-09-04 06:02:45,231 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting 4a5fd927b9664c57bdcf0ebbbe62693c, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=150, earliestPutTs=1693807355112 2023-09-04 06:02:45,244 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] throttle.PressureAwareThroughputController(145): c729f1dd7f7ce1a44f1822ce19e5bd40#info#compaction#29 average throughput is 28.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 06:02:45,261 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/fa072c8297194732b4df1acbb3d5784d as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/fa072c8297194732b4df1acbb3d5784d 2023-09-04 06:02:45,267 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in c729f1dd7f7ce1a44f1822ce19e5bd40/info of c729f1dd7f7ce1a44f1822ce19e5bd40 into fa072c8297194732b4df1acbb3d5784d(size=62.7 K), total size for store is 62.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 06:02:45,267 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:02:45,267 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40., storeName=c729f1dd7f7ce1a44f1822ce19e5bd40/info, priority=13, startTime=1693807365228; duration=0sec 2023-09-04 06:02:45,267 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:02:55,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:02:55,246 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing c729f1dd7f7ce1a44f1822ce19e5bd40 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 06:02:55,262 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=162 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/c91edacc4b604d65b931e4fd38cd3338 2023-09-04 06:02:55,268 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/c91edacc4b604d65b931e4fd38cd3338 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/c91edacc4b604d65b931e4fd38cd3338 2023-09-04 06:02:55,275 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/c91edacc4b604d65b931e4fd38cd3338, entries=8, sequenceid=162, filesize=13.0 K 2023-09-04 06:02:55,276 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=1.03 KB/1051 for c729f1dd7f7ce1a44f1822ce19e5bd40 in 30ms, sequenceid=162, compaction requested=false 2023-09-04 06:02:55,276 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:02:57,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:02:57,257 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing c729f1dd7f7ce1a44f1822ce19e5bd40 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 06:02:57,274 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=173 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/b130e912cc594f3398653fdf17c69848 2023-09-04 06:02:57,281 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/b130e912cc594f3398653fdf17c69848 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/b130e912cc594f3398653fdf17c69848 2023-09-04 06:02:57,287 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 06:02:57,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] ipc.CallRunner(144): callId: 179 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:48626 deadline: 1693807387287, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:02:57,288 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/b130e912cc594f3398653fdf17c69848, entries=8, sequenceid=173, filesize=13.0 K 2023-09-04 06:02:57,289 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for c729f1dd7f7ce1a44f1822ce19e5bd40 in 32ms, sequenceid=173, compaction requested=true 2023-09-04 06:02:57,289 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:02:57,289 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:02:57,289 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 06:02:57,290 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 90753 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 06:02:57,290 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1912): c729f1dd7f7ce1a44f1822ce19e5bd40/info is initiating minor compaction (all files) 2023-09-04 06:02:57,291 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c729f1dd7f7ce1a44f1822ce19e5bd40/info in TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:02:57,291 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/fa072c8297194732b4df1acbb3d5784d, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/c91edacc4b604d65b931e4fd38cd3338, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/b130e912cc594f3398653fdf17c69848] into tmpdir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp, totalSize=88.6 K 2023-09-04 06:02:57,291 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting fa072c8297194732b4df1acbb3d5784d, keycount=56, bloomtype=ROW, size=62.7 K, encoding=NONE, compression=NONE, seqNum=150, earliestPutTs=1693807320856 2023-09-04 06:02:57,292 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting c91edacc4b604d65b931e4fd38cd3338, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=162, earliestPutTs=1693807365200 2023-09-04 06:02:57,292 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting b130e912cc594f3398653fdf17c69848, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=173, earliestPutTs=1693807375247 2023-09-04 06:02:57,320 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] throttle.PressureAwareThroughputController(145): c729f1dd7f7ce1a44f1822ce19e5bd40#info#compaction#32 average throughput is 24.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 06:02:57,338 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/048a61f10b9a4c278df8af05a9c9dca2 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/048a61f10b9a4c278df8af05a9c9dca2 2023-09-04 06:02:57,346 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in c729f1dd7f7ce1a44f1822ce19e5bd40/info of c729f1dd7f7ce1a44f1822ce19e5bd40 into 048a61f10b9a4c278df8af05a9c9dca2(size=79.3 K), total size for store is 79.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 06:02:57,346 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:02:57,346 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40., storeName=c729f1dd7f7ce1a44f1822ce19e5bd40/info, priority=13, startTime=1693807377289; duration=0sec 2023-09-04 06:02:57,346 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:03:07,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:03:07,328 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing c729f1dd7f7ce1a44f1822ce19e5bd40 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-04 06:03:07,352 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=199 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/453e15c0ba214621af857227fd6df546 2023-09-04 06:03:07,359 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/453e15c0ba214621af857227fd6df546 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/453e15c0ba214621af857227fd6df546 2023-09-04 06:03:07,365 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/453e15c0ba214621af857227fd6df546, entries=22, sequenceid=199, filesize=27.4 K 2023-09-04 06:03:07,366 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=3.08 KB/3153 for c729f1dd7f7ce1a44f1822ce19e5bd40 in 38ms, sequenceid=199, compaction requested=false 2023-09-04 06:03:07,366 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:03:09,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:03:09,343 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing c729f1dd7f7ce1a44f1822ce19e5bd40 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 06:03:09,355 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=210 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/f407e24deec6491db798303e8b86ae03 2023-09-04 06:03:09,370 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 06:03:09,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] ipc.CallRunner(144): callId: 211 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:48626 deadline: 1693807399370, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:03:09,374 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/f407e24deec6491db798303e8b86ae03 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/f407e24deec6491db798303e8b86ae03 2023-09-04 06:03:09,386 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/f407e24deec6491db798303e8b86ae03, entries=8, sequenceid=210, filesize=13.0 K 2023-09-04 06:03:09,387 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for c729f1dd7f7ce1a44f1822ce19e5bd40 in 44ms, sequenceid=210, compaction requested=true 2023-09-04 06:03:09,387 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:03:09,388 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-09-04 06:03:09,388 DEBUG [RS:0;jenkins-hbase4:44413-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 06:03:09,389 DEBUG [RS:0;jenkins-hbase4:44413-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 122538 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 06:03:09,390 DEBUG [RS:0;jenkins-hbase4:44413-longCompactions-0] regionserver.HStore(1912): c729f1dd7f7ce1a44f1822ce19e5bd40/info is initiating minor compaction (all files) 2023-09-04 06:03:09,390 INFO [RS:0;jenkins-hbase4:44413-longCompactions-0] regionserver.HRegion(2259): Starting compaction of c729f1dd7f7ce1a44f1822ce19e5bd40/info in TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:03:09,390 INFO [RS:0;jenkins-hbase4:44413-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/048a61f10b9a4c278df8af05a9c9dca2, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/453e15c0ba214621af857227fd6df546, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/f407e24deec6491db798303e8b86ae03] into tmpdir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp, totalSize=119.7 K 2023-09-04 06:03:09,390 DEBUG [RS:0;jenkins-hbase4:44413-longCompactions-0] compactions.Compactor(207): Compacting 048a61f10b9a4c278df8af05a9c9dca2, keycount=72, bloomtype=ROW, size=79.3 K, encoding=NONE, compression=NONE, seqNum=173, earliestPutTs=1693807320856 2023-09-04 06:03:09,391 DEBUG [RS:0;jenkins-hbase4:44413-longCompactions-0] compactions.Compactor(207): Compacting 453e15c0ba214621af857227fd6df546, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=199, earliestPutTs=1693807377257 2023-09-04 06:03:09,392 DEBUG [RS:0;jenkins-hbase4:44413-longCompactions-0] compactions.Compactor(207): Compacting f407e24deec6491db798303e8b86ae03, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=210, earliestPutTs=1693807387330 2023-09-04 06:03:09,422 INFO [RS:0;jenkins-hbase4:44413-longCompactions-0] throttle.PressureAwareThroughputController(145): c729f1dd7f7ce1a44f1822ce19e5bd40#info#compaction#35 average throughput is 17.04 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 06:03:09,445 DEBUG [RS:0;jenkins-hbase4:44413-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/28b4caeb104846b39d42d3391ee53964 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/28b4caeb104846b39d42d3391ee53964 2023-09-04 06:03:09,453 INFO [RS:0;jenkins-hbase4:44413-longCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in c729f1dd7f7ce1a44f1822ce19e5bd40/info of c729f1dd7f7ce1a44f1822ce19e5bd40 into 28b4caeb104846b39d42d3391ee53964(size=110.3 K), total size for store is 110.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 06:03:09,453 DEBUG [RS:0;jenkins-hbase4:44413-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:03:09,453 INFO [RS:0;jenkins-hbase4:44413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40., storeName=c729f1dd7f7ce1a44f1822ce19e5bd40/info, priority=13, startTime=1693807389388; duration=0sec 2023-09-04 06:03:09,453 DEBUG [RS:0;jenkins-hbase4:44413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:03:19,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:03:19,396 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing c729f1dd7f7ce1a44f1822ce19e5bd40 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-04 06:03:19,409 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=236 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/3810c47e27c640c99d3b35b3c2cb901f 2023-09-04 06:03:19,415 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/3810c47e27c640c99d3b35b3c2cb901f as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/3810c47e27c640c99d3b35b3c2cb901f 2023-09-04 06:03:19,420 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/3810c47e27c640c99d3b35b3c2cb901f, entries=22, sequenceid=236, filesize=27.4 K 2023-09-04 06:03:19,421 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=5.13 KB/5255 for c729f1dd7f7ce1a44f1822ce19e5bd40 in 25ms, sequenceid=236, compaction requested=false 2023-09-04 06:03:19,421 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:03:21,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:03:21,410 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing c729f1dd7f7ce1a44f1822ce19e5bd40 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 06:03:21,451 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=247 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/2e3fcf91d010407fb89a6a30eed4820a 2023-09-04 06:03:21,460 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/2e3fcf91d010407fb89a6a30eed4820a as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/2e3fcf91d010407fb89a6a30eed4820a 2023-09-04 06:03:21,466 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 06:03:21,466 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] ipc.CallRunner(144): callId: 243 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:48626 deadline: 1693807411466, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:03:21,471 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/2e3fcf91d010407fb89a6a30eed4820a, entries=8, sequenceid=247, filesize=13.0 K 2023-09-04 06:03:21,472 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for c729f1dd7f7ce1a44f1822ce19e5bd40 in 62ms, sequenceid=247, compaction requested=true 2023-09-04 06:03:21,472 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:03:21,472 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:03:21,472 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 06:03:21,474 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 154238 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 06:03:21,474 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1912): c729f1dd7f7ce1a44f1822ce19e5bd40/info is initiating minor compaction (all files) 2023-09-04 06:03:21,474 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c729f1dd7f7ce1a44f1822ce19e5bd40/info in TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:03:21,474 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/28b4caeb104846b39d42d3391ee53964, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/3810c47e27c640c99d3b35b3c2cb901f, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/2e3fcf91d010407fb89a6a30eed4820a] into tmpdir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp, totalSize=150.6 K 2023-09-04 06:03:21,475 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting 28b4caeb104846b39d42d3391ee53964, keycount=102, bloomtype=ROW, size=110.3 K, encoding=NONE, compression=NONE, seqNum=210, earliestPutTs=1693807320856 2023-09-04 06:03:21,476 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting 3810c47e27c640c99d3b35b3c2cb901f, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=236, earliestPutTs=1693807389343 2023-09-04 06:03:21,477 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting 2e3fcf91d010407fb89a6a30eed4820a, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=247, earliestPutTs=1693807399397 2023-09-04 06:03:21,502 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] throttle.PressureAwareThroughputController(145): c729f1dd7f7ce1a44f1822ce19e5bd40#info#compaction#38 average throughput is 33.08 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 06:03:21,541 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/05cf6f127bb04202a3d620c1dae2323d as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/05cf6f127bb04202a3d620c1dae2323d 2023-09-04 06:03:21,549 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in c729f1dd7f7ce1a44f1822ce19e5bd40/info of c729f1dd7f7ce1a44f1822ce19e5bd40 into 05cf6f127bb04202a3d620c1dae2323d(size=141.4 K), total size for store is 141.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 06:03:21,549 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:03:21,549 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40., storeName=c729f1dd7f7ce1a44f1822ce19e5bd40/info, priority=13, startTime=1693807401472; duration=0sec 2023-09-04 06:03:21,550 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:03:27,758 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c2695740c88d7b4dd012810de5537468/info/e7a41f39b10b4f1382ba5803301ef10a.999967069604f4f8061d4a09952cbc92->hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/e7a41f39b10b4f1382ba5803301ef10a-bottom] to archive 2023-09-04 06:03:27,759 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-09-04 06:03:27,761 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c2695740c88d7b4dd012810de5537468/info/e7a41f39b10b4f1382ba5803301ef10a.999967069604f4f8061d4a09952cbc92 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c2695740c88d7b4dd012810de5537468/info/e7a41f39b10b4f1382ba5803301ef10a.999967069604f4f8061d4a09952cbc92 2023-09-04 06:03:27,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e7a41f39b10b4f1382ba5803301ef10a.999967069604f4f8061d4a09952cbc92->hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/999967069604f4f8061d4a09952cbc92/info/e7a41f39b10b4f1382ba5803301ef10a-top, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e96f84b2c0f84929a0e6315a70d5ca18, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/TestLogRolling-testLogRolling=999967069604f4f8061d4a09952cbc92-5cacbf7fe3864061b8a9e84eeab731ed, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e73daf21f30342d79a91e798dcb6403b, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/fa072c8297194732b4df1acbb3d5784d, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/4a5fd927b9664c57bdcf0ebbbe62693c, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/c91edacc4b604d65b931e4fd38cd3338, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/048a61f10b9a4c278df8af05a9c9dca2, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/b130e912cc594f3398653fdf17c69848, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/453e15c0ba214621af857227fd6df546, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/28b4caeb104846b39d42d3391ee53964, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/f407e24deec6491db798303e8b86ae03, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/3810c47e27c640c99d3b35b3c2cb901f, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/2e3fcf91d010407fb89a6a30eed4820a] to archive 2023-09-04 06:03:27,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-09-04 06:03:27,775 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e7a41f39b10b4f1382ba5803301ef10a.999967069604f4f8061d4a09952cbc92 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e7a41f39b10b4f1382ba5803301ef10a.999967069604f4f8061d4a09952cbc92 2023-09-04 06:03:27,777 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e96f84b2c0f84929a0e6315a70d5ca18 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e96f84b2c0f84929a0e6315a70d5ca18 2023-09-04 06:03:27,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/TestLogRolling-testLogRolling=999967069604f4f8061d4a09952cbc92-5cacbf7fe3864061b8a9e84eeab731ed to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/TestLogRolling-testLogRolling=999967069604f4f8061d4a09952cbc92-5cacbf7fe3864061b8a9e84eeab731ed 2023-09-04 06:03:27,779 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e73daf21f30342d79a91e798dcb6403b to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/e73daf21f30342d79a91e798dcb6403b 2023-09-04 06:03:27,781 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/fa072c8297194732b4df1acbb3d5784d to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/fa072c8297194732b4df1acbb3d5784d 2023-09-04 06:03:27,782 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/4a5fd927b9664c57bdcf0ebbbe62693c to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/4a5fd927b9664c57bdcf0ebbbe62693c 2023-09-04 06:03:27,783 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/c91edacc4b604d65b931e4fd38cd3338 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/c91edacc4b604d65b931e4fd38cd3338 2023-09-04 06:03:27,785 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/048a61f10b9a4c278df8af05a9c9dca2 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/048a61f10b9a4c278df8af05a9c9dca2 2023-09-04 06:03:27,786 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/b130e912cc594f3398653fdf17c69848 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/b130e912cc594f3398653fdf17c69848 2023-09-04 06:03:27,787 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/453e15c0ba214621af857227fd6df546 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/453e15c0ba214621af857227fd6df546 2023-09-04 06:03:27,788 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/28b4caeb104846b39d42d3391ee53964 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/28b4caeb104846b39d42d3391ee53964 2023-09-04 06:03:27,789 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/f407e24deec6491db798303e8b86ae03 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/f407e24deec6491db798303e8b86ae03 2023-09-04 06:03:27,790 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/3810c47e27c640c99d3b35b3c2cb901f to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/3810c47e27c640c99d3b35b3c2cb901f 2023-09-04 06:03:27,792 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/2e3fcf91d010407fb89a6a30eed4820a to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/2e3fcf91d010407fb89a6a30eed4820a 2023-09-04 06:03:31,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:03:31,516 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing c729f1dd7f7ce1a44f1822ce19e5bd40 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-04 06:03:31,531 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=273 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/eb29b331bbda46468152347018bc436a 2023-09-04 06:03:31,538 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/eb29b331bbda46468152347018bc436a as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/eb29b331bbda46468152347018bc436a 2023-09-04 06:03:31,543 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/eb29b331bbda46468152347018bc436a, entries=22, sequenceid=273, filesize=27.4 K 2023-09-04 06:03:31,544 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=7.18 KB/7357 for c729f1dd7f7ce1a44f1822ce19e5bd40 in 28ms, sequenceid=273, compaction requested=false 2023-09-04 06:03:31,545 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:03:33,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:03:33,531 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing c729f1dd7f7ce1a44f1822ce19e5bd40 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 06:03:33,553 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=284 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/0863795bb7774263a3cb86fa8084251b 2023-09-04 06:03:33,561 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/0863795bb7774263a3cb86fa8084251b as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/0863795bb7774263a3cb86fa8084251b 2023-09-04 06:03:33,562 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 06:03:33,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] ipc.CallRunner(144): callId: 275 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:48626 deadline: 1693807423561, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:03:33,568 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/0863795bb7774263a3cb86fa8084251b, entries=8, sequenceid=284, filesize=13.0 K 2023-09-04 06:03:33,569 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for c729f1dd7f7ce1a44f1822ce19e5bd40 in 39ms, sequenceid=284, compaction requested=true 2023-09-04 06:03:33,569 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:03:33,569 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:03:33,569 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 06:03:33,572 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 186157 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 06:03:33,572 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1912): c729f1dd7f7ce1a44f1822ce19e5bd40/info is initiating minor compaction (all files) 2023-09-04 06:03:33,572 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c729f1dd7f7ce1a44f1822ce19e5bd40/info in TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:03:33,572 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/05cf6f127bb04202a3d620c1dae2323d, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/eb29b331bbda46468152347018bc436a, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/0863795bb7774263a3cb86fa8084251b] into tmpdir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp, totalSize=181.8 K 2023-09-04 06:03:33,573 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting 05cf6f127bb04202a3d620c1dae2323d, keycount=132, bloomtype=ROW, size=141.4 K, encoding=NONE, compression=NONE, seqNum=247, earliestPutTs=1693807320856 2023-09-04 06:03:33,574 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting eb29b331bbda46468152347018bc436a, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=273, earliestPutTs=1693807401411 2023-09-04 06:03:33,574 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting 0863795bb7774263a3cb86fa8084251b, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=284, earliestPutTs=1693807411518 2023-09-04 06:03:33,613 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] throttle.PressureAwareThroughputController(145): c729f1dd7f7ce1a44f1822ce19e5bd40#info#compaction#41 average throughput is 32.47 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 06:03:33,647 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/2b7f2bc41bf0483e85375cd7eb13a7f3 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/2b7f2bc41bf0483e85375cd7eb13a7f3 2023-09-04 06:03:33,653 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in c729f1dd7f7ce1a44f1822ce19e5bd40/info of c729f1dd7f7ce1a44f1822ce19e5bd40 into 2b7f2bc41bf0483e85375cd7eb13a7f3(size=171.8 K), total size for store is 171.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 06:03:33,653 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:03:33,653 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40., storeName=c729f1dd7f7ce1a44f1822ce19e5bd40/info, priority=13, startTime=1693807413569; duration=0sec 2023-09-04 06:03:33,653 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:03:43,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:03:43,655 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing c729f1dd7f7ce1a44f1822ce19e5bd40 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-04 06:03:43,668 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 06:03:43,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] ipc.CallRunner(144): callId: 285 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:48626 deadline: 1693807433668, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=c729f1dd7f7ce1a44f1822ce19e5bd40, server=jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:03:43,670 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=310 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/8b5517d337274b7182b8513a7d6031aa 2023-09-04 06:03:43,682 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/8b5517d337274b7182b8513a7d6031aa as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/8b5517d337274b7182b8513a7d6031aa 2023-09-04 06:03:43,689 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/8b5517d337274b7182b8513a7d6031aa, entries=22, sequenceid=310, filesize=27.4 K 2023-09-04 06:03:43,690 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=7.18 KB/7357 for c729f1dd7f7ce1a44f1822ce19e5bd40 in 35ms, sequenceid=310, compaction requested=false 2023-09-04 06:03:43,690 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:03:53,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44413] regionserver.HRegion(9158): Flush requested on c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:03:53,726 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing c729f1dd7f7ce1a44f1822ce19e5bd40 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 06:03:53,740 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=321 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/6b3d161f4e1448108fea73a980a36ca0 2023-09-04 06:03:53,746 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/6b3d161f4e1448108fea73a980a36ca0 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/6b3d161f4e1448108fea73a980a36ca0 2023-09-04 06:03:53,751 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/6b3d161f4e1448108fea73a980a36ca0, entries=8, sequenceid=321, filesize=13.0 K 2023-09-04 06:03:53,752 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=1.03 KB/1051 for c729f1dd7f7ce1a44f1822ce19e5bd40 in 26ms, sequenceid=321, compaction requested=true 2023-09-04 06:03:53,753 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:03:53,753 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:03:53,753 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 06:03:53,754 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 217314 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 06:03:53,754 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1912): c729f1dd7f7ce1a44f1822ce19e5bd40/info is initiating minor compaction (all files) 2023-09-04 06:03:53,754 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c729f1dd7f7ce1a44f1822ce19e5bd40/info in TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:03:53,754 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/2b7f2bc41bf0483e85375cd7eb13a7f3, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/8b5517d337274b7182b8513a7d6031aa, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/6b3d161f4e1448108fea73a980a36ca0] into tmpdir=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp, totalSize=212.2 K 2023-09-04 06:03:53,755 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting 2b7f2bc41bf0483e85375cd7eb13a7f3, keycount=162, bloomtype=ROW, size=171.8 K, encoding=NONE, compression=NONE, seqNum=284, earliestPutTs=1693807320856 2023-09-04 06:03:53,755 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting 8b5517d337274b7182b8513a7d6031aa, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=310, earliestPutTs=1693807413531 2023-09-04 06:03:53,755 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] compactions.Compactor(207): Compacting 6b3d161f4e1448108fea73a980a36ca0, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=321, earliestPutTs=1693807423656 2023-09-04 06:03:53,774 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] throttle.PressureAwareThroughputController(145): c729f1dd7f7ce1a44f1822ce19e5bd40#info#compaction#44 average throughput is 38.49 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 06:03:53,789 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/d5451f4cd68648c99b8c482e4b9afff7 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/d5451f4cd68648c99b8c482e4b9afff7 2023-09-04 06:03:53,795 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in c729f1dd7f7ce1a44f1822ce19e5bd40/info of c729f1dd7f7ce1a44f1822ce19e5bd40 into d5451f4cd68648c99b8c482e4b9afff7(size=202.9 K), total size for store is 202.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 06:03:53,795 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:03:53,795 INFO [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40., storeName=c729f1dd7f7ce1a44f1822ce19e5bd40/info, priority=13, startTime=1693807433753; duration=0sec 2023-09-04 06:03:53,796 DEBUG [RS:0;jenkins-hbase4:44413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 06:03:55,728 INFO [Listener at localhost/37417] wal.AbstractTestLogRolling(188): after writing there are 0 log files 2023-09-04 06:03:55,746 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37037,DS-d3fb595d-6b6c-4a03-a377-2f38ea57a3d3,DISK] 2023-09-04 06:03:55,747 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43733,DS-34087bb1-246a-42ca-b408-b7b12824778b,DISK] 2023-09-04 06:03:55,750 INFO [Listener at localhost/37417] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/WALs/jenkins-hbase4.apache.org,44413,1693807287501/jenkins-hbase4.apache.org%2C44413%2C1693807287501.1693807287883 with entries=309, filesize=300.61 KB; new WAL /user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/WALs/jenkins-hbase4.apache.org,44413,1693807287501/jenkins-hbase4.apache.org%2C44413%2C1693807287501.1693807435728 2023-09-04 06:03:55,750 DEBUG [Listener at localhost/37417] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37037,DS-d3fb595d-6b6c-4a03-a377-2f38ea57a3d3,DISK], DatanodeInfoWithStorage[127.0.0.1:43733,DS-34087bb1-246a-42ca-b408-b7b12824778b,DISK]] 2023-09-04 06:03:55,750 DEBUG [Listener at localhost/37417] wal.AbstractFSWAL(716): hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/WALs/jenkins-hbase4.apache.org,44413,1693807287501/jenkins-hbase4.apache.org%2C44413%2C1693807287501.1693807287883 is not closed yet, will try archiving it next time 2023-09-04 06:03:55,756 DEBUG [Listener at localhost/37417] regionserver.HRegion(2446): Flush status journal for c2695740c88d7b4dd012810de5537468: 2023-09-04 06:03:55,756 INFO [Listener at localhost/37417] regionserver.HRegion(2745): Flushing f33565f139fdc8b1ac198254ec83c339 1/1 column families, dataSize=78 B heapSize=488 B 2023-09-04 06:03:55,767 INFO [Listener at localhost/37417] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/namespace/f33565f139fdc8b1ac198254ec83c339/.tmp/info/b75da98a5833434f80c3e1ad80199f84 2023-09-04 06:03:55,773 DEBUG [Listener at localhost/37417] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/namespace/f33565f139fdc8b1ac198254ec83c339/.tmp/info/b75da98a5833434f80c3e1ad80199f84 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/namespace/f33565f139fdc8b1ac198254ec83c339/info/b75da98a5833434f80c3e1ad80199f84 2023-09-04 06:03:55,779 INFO [Listener at localhost/37417] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/namespace/f33565f139fdc8b1ac198254ec83c339/info/b75da98a5833434f80c3e1ad80199f84, entries=2, sequenceid=6, filesize=4.8 K 2023-09-04 06:03:55,779 INFO [Listener at localhost/37417] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for f33565f139fdc8b1ac198254ec83c339 in 23ms, sequenceid=6, compaction requested=false 2023-09-04 06:03:55,780 DEBUG [Listener at localhost/37417] regionserver.HRegion(2446): Flush status journal for f33565f139fdc8b1ac198254ec83c339: 2023-09-04 06:03:55,780 INFO [Listener at localhost/37417] regionserver.HRegion(2745): Flushing c729f1dd7f7ce1a44f1822ce19e5bd40 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-04 06:03:55,793 INFO [Listener at localhost/37417] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=326 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/426e7fe8329744e18805ed01927fa24b 2023-09-04 06:03:55,799 DEBUG [Listener at localhost/37417] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/.tmp/info/426e7fe8329744e18805ed01927fa24b as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/426e7fe8329744e18805ed01927fa24b 2023-09-04 06:03:55,805 INFO [Listener at localhost/37417] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/426e7fe8329744e18805ed01927fa24b, entries=1, sequenceid=326, filesize=5.8 K 2023-09-04 06:03:55,806 INFO [Listener at localhost/37417] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for c729f1dd7f7ce1a44f1822ce19e5bd40 in 26ms, sequenceid=326, compaction requested=false 2023-09-04 06:03:55,806 DEBUG [Listener at localhost/37417] regionserver.HRegion(2446): Flush status journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:03:55,806 INFO [Listener at localhost/37417] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.26 KB heapSize=4.19 KB 2023-09-04 06:03:55,823 INFO [Listener at localhost/37417] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.26 KB at sequenceid=24 (bloomFilter=false), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/.tmp/info/5033de3d22b440e7ac33b57d9b7eb513 2023-09-04 06:03:55,829 DEBUG [Listener at localhost/37417] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/.tmp/info/5033de3d22b440e7ac33b57d9b7eb513 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/info/5033de3d22b440e7ac33b57d9b7eb513 2023-09-04 06:03:55,834 INFO [Listener at localhost/37417] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/info/5033de3d22b440e7ac33b57d9b7eb513, entries=16, sequenceid=24, filesize=7.0 K 2023-09-04 06:03:55,834 INFO [Listener at localhost/37417] regionserver.HRegion(2948): Finished flush of dataSize ~2.26 KB/2312, heapSize ~3.67 KB/3760, currentSize=0 B/0 for 1588230740 in 28ms, sequenceid=24, compaction requested=false 2023-09-04 06:03:55,835 DEBUG [Listener at localhost/37417] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-09-04 06:03:55,851 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43733,DS-34087bb1-246a-42ca-b408-b7b12824778b,DISK] 2023-09-04 06:03:55,851 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37037,DS-d3fb595d-6b6c-4a03-a377-2f38ea57a3d3,DISK] 2023-09-04 06:03:55,854 INFO [Listener at localhost/37417] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/WALs/jenkins-hbase4.apache.org,44413,1693807287501/jenkins-hbase4.apache.org%2C44413%2C1693807287501.1693807435728 with entries=4, filesize=1.22 KB; new WAL /user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/WALs/jenkins-hbase4.apache.org,44413,1693807287501/jenkins-hbase4.apache.org%2C44413%2C1693807287501.1693807435835 2023-09-04 06:03:55,854 DEBUG [Listener at localhost/37417] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43733,DS-34087bb1-246a-42ca-b408-b7b12824778b,DISK], DatanodeInfoWithStorage[127.0.0.1:37037,DS-d3fb595d-6b6c-4a03-a377-2f38ea57a3d3,DISK]] 2023-09-04 06:03:55,854 DEBUG [Listener at localhost/37417] wal.AbstractFSWAL(716): hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/WALs/jenkins-hbase4.apache.org,44413,1693807287501/jenkins-hbase4.apache.org%2C44413%2C1693807287501.1693807435728 is not closed yet, will try archiving it next time 2023-09-04 06:03:55,855 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/WALs/jenkins-hbase4.apache.org,44413,1693807287501/jenkins-hbase4.apache.org%2C44413%2C1693807287501.1693807287883 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/oldWALs/jenkins-hbase4.apache.org%2C44413%2C1693807287501.1693807287883 2023-09-04 06:03:55,856 INFO [Listener at localhost/37417] hbase.Waiter(180): Waiting up to [5,000] milli-secs(wait.for.ratio=[1]) 2023-09-04 06:03:55,857 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/WALs/jenkins-hbase4.apache.org,44413,1693807287501/jenkins-hbase4.apache.org%2C44413%2C1693807287501.1693807435728 not finished, retry = 0 2023-09-04 06:03:55,958 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/WALs/jenkins-hbase4.apache.org,44413,1693807287501/jenkins-hbase4.apache.org%2C44413%2C1693807287501.1693807435728 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/oldWALs/jenkins-hbase4.apache.org%2C44413%2C1693807287501.1693807435728 2023-09-04 06:03:56,056 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-09-04 06:03:56,056 INFO [Listener at localhost/37417] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-09-04 06:03:56,056 DEBUG [Listener at localhost/37417] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x57b23b62 to 127.0.0.1:56951 2023-09-04 06:03:56,056 DEBUG [Listener at localhost/37417] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:03:56,056 DEBUG [Listener at localhost/37417] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-09-04 06:03:56,056 DEBUG [Listener at localhost/37417] util.JVMClusterUtil(257): Found active master hash=1872582880, stopped=false 2023-09-04 06:03:56,056 INFO [Listener at localhost/37417] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,46229,1693807287459 2023-09-04 06:03:56,058 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 06:03:56,058 INFO [Listener at localhost/37417] procedure2.ProcedureExecutor(628): Stopping 2023-09-04 06:03:56,058 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:03:56,058 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): regionserver:44413-0x100758754360001, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 06:03:56,059 DEBUG [Listener at localhost/37417] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x467c2c8c to 127.0.0.1:56951 2023-09-04 06:03:56,059 DEBUG [Listener at localhost/37417] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:03:56,059 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:03:56,059 INFO [Listener at localhost/37417] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,44413,1693807287501' ***** 2023-09-04 06:03:56,059 INFO [Listener at localhost/37417] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-09-04 06:03:56,059 INFO [RS:0;jenkins-hbase4:44413] regionserver.HeapMemoryManager(220): Stopping 2023-09-04 06:03:56,059 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44413-0x100758754360001, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:03:56,060 INFO [RS:0;jenkins-hbase4:44413] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-09-04 06:03:56,060 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-09-04 06:03:56,060 INFO [RS:0;jenkins-hbase4:44413] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-09-04 06:03:56,060 INFO [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(3305): Received CLOSE for c2695740c88d7b4dd012810de5537468 2023-09-04 06:03:56,060 INFO [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(3305): Received CLOSE for f33565f139fdc8b1ac198254ec83c339 2023-09-04 06:03:56,060 INFO [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(3305): Received CLOSE for c729f1dd7f7ce1a44f1822ce19e5bd40 2023-09-04 06:03:56,060 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c2695740c88d7b4dd012810de5537468, disabling compactions & flushes 2023-09-04 06:03:56,060 INFO [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:03:56,060 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468. 2023-09-04 06:03:56,060 DEBUG [RS:0;jenkins-hbase4:44413] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x60f1984e to 127.0.0.1:56951 2023-09-04 06:03:56,060 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468. 2023-09-04 06:03:56,060 DEBUG [RS:0;jenkins-hbase4:44413] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:03:56,060 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468. after waiting 0 ms 2023-09-04 06:03:56,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468. 2023-09-04 06:03:56,061 INFO [RS:0;jenkins-hbase4:44413] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-09-04 06:03:56,061 INFO [RS:0;jenkins-hbase4:44413] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-09-04 06:03:56,061 INFO [RS:0;jenkins-hbase4:44413] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-09-04 06:03:56,061 INFO [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-09-04 06:03:56,062 INFO [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-09-04 06:03:56,062 DEBUG [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(1478): Online Regions={c2695740c88d7b4dd012810de5537468=TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468., f33565f139fdc8b1ac198254ec83c339=hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339., c729f1dd7f7ce1a44f1822ce19e5bd40=TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40., 1588230740=hbase:meta,,1.1588230740} 2023-09-04 06:03:56,062 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 06:03:56,062 DEBUG [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(1504): Waiting on 1588230740, c2695740c88d7b4dd012810de5537468, c729f1dd7f7ce1a44f1822ce19e5bd40, f33565f139fdc8b1ac198254ec83c339 2023-09-04 06:03:56,063 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 06:03:56,063 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 06:03:56,063 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 06:03:56,063 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 06:03:56,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c2695740c88d7b4dd012810de5537468/recovered.edits/118.seqid, newMaxSeqId=118, maxSeqId=113 2023-09-04 06:03:56,070 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468. 2023-09-04 06:03:56,070 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/meta/1588230740/recovered.edits/27.seqid, newMaxSeqId=27, maxSeqId=1 2023-09-04 06:03:56,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c2695740c88d7b4dd012810de5537468: 2023-09-04 06:03:56,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,,1693807322938.c2695740c88d7b4dd012810de5537468. 2023-09-04 06:03:56,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f33565f139fdc8b1ac198254ec83c339, disabling compactions & flushes 2023-09-04 06:03:56,070 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339. 2023-09-04 06:03:56,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339. 2023-09-04 06:03:56,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339. after waiting 0 ms 2023-09-04 06:03:56,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339. 2023-09-04 06:03:56,070 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-04 06:03:56,121 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 06:03:56,121 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 06:03:56,122 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-09-04 06:03:56,127 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/hbase/namespace/f33565f139fdc8b1ac198254ec83c339/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-09-04 06:03:56,128 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339. 2023-09-04 06:03:56,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f33565f139fdc8b1ac198254ec83c339: 2023-09-04 06:03:56,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693807288088.f33565f139fdc8b1ac198254ec83c339. 2023-09-04 06:03:56,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c729f1dd7f7ce1a44f1822ce19e5bd40, disabling compactions & flushes 2023-09-04 06:03:56,129 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:03:56,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:03:56,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. after waiting 0 ms 2023-09-04 06:03:56,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:03:56,129 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/05cf6f127bb04202a3d620c1dae2323d, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/eb29b331bbda46468152347018bc436a, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/2b7f2bc41bf0483e85375cd7eb13a7f3, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/0863795bb7774263a3cb86fa8084251b, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/8b5517d337274b7182b8513a7d6031aa, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/6b3d161f4e1448108fea73a980a36ca0] to archive 2023-09-04 06:03:56,130 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-09-04 06:03:56,131 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/05cf6f127bb04202a3d620c1dae2323d to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/05cf6f127bb04202a3d620c1dae2323d 2023-09-04 06:03:56,132 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/eb29b331bbda46468152347018bc436a to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/eb29b331bbda46468152347018bc436a 2023-09-04 06:03:56,134 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/2b7f2bc41bf0483e85375cd7eb13a7f3 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/2b7f2bc41bf0483e85375cd7eb13a7f3 2023-09-04 06:03:56,135 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/0863795bb7774263a3cb86fa8084251b to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/0863795bb7774263a3cb86fa8084251b 2023-09-04 06:03:56,136 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/8b5517d337274b7182b8513a7d6031aa to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/8b5517d337274b7182b8513a7d6031aa 2023-09-04 06:03:56,138 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/6b3d161f4e1448108fea73a980a36ca0 to hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/archive/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/info/6b3d161f4e1448108fea73a980a36ca0 2023-09-04 06:03:56,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/data/default/TestLogRolling-testLogRolling/c729f1dd7f7ce1a44f1822ce19e5bd40/recovered.edits/329.seqid, newMaxSeqId=329, maxSeqId=113 2023-09-04 06:03:56,146 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:03:56,146 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c729f1dd7f7ce1a44f1822ce19e5bd40: 2023-09-04 06:03:56,146 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,row0064,1693807322938.c729f1dd7f7ce1a44f1822ce19e5bd40. 2023-09-04 06:03:56,263 INFO [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,44413,1693807287501; all regions closed. 2023-09-04 06:03:56,269 DEBUG [RS:0;jenkins-hbase4:44413] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/oldWALs 2023-09-04 06:03:56,269 INFO [RS:0;jenkins-hbase4:44413] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C44413%2C1693807287501.meta:.meta(num 1693807288030) 2023-09-04 06:03:56,274 DEBUG [RS:0;jenkins-hbase4:44413] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/oldWALs 2023-09-04 06:03:56,274 INFO [RS:0;jenkins-hbase4:44413] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C44413%2C1693807287501:(num 1693807435835) 2023-09-04 06:03:56,274 DEBUG [RS:0;jenkins-hbase4:44413] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:03:56,274 INFO [RS:0;jenkins-hbase4:44413] regionserver.LeaseManager(133): Closed leases 2023-09-04 06:03:56,274 INFO [RS:0;jenkins-hbase4:44413] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-09-04 06:03:56,274 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 06:03:56,275 INFO [RS:0;jenkins-hbase4:44413] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44413 2023-09-04 06:03:56,278 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): regionserver:44413-0x100758754360001, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,44413,1693807287501 2023-09-04 06:03:56,278 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 06:03:56,278 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): regionserver:44413-0x100758754360001, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 06:03:56,279 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,44413,1693807287501] 2023-09-04 06:03:56,280 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,44413,1693807287501; numProcessing=1 2023-09-04 06:03:56,282 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,44413,1693807287501 already deleted, retry=false 2023-09-04 06:03:56,282 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,44413,1693807287501 expired; onlineServers=0 2023-09-04 06:03:56,282 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46229,1693807287459' ***** 2023-09-04 06:03:56,282 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-09-04 06:03:56,282 DEBUG [M:0;jenkins-hbase4:46229] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6ba2c1cb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 06:03:56,282 INFO [M:0;jenkins-hbase4:46229] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46229,1693807287459 2023-09-04 06:03:56,282 INFO [M:0;jenkins-hbase4:46229] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46229,1693807287459; all regions closed. 2023-09-04 06:03:56,282 DEBUG [M:0;jenkins-hbase4:46229] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:03:56,282 DEBUG [M:0;jenkins-hbase4:46229] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-09-04 06:03:56,282 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-09-04 06:03:56,282 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693807287666] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693807287666,5,FailOnTimeoutGroup] 2023-09-04 06:03:56,282 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693807287667] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693807287667,5,FailOnTimeoutGroup] 2023-09-04 06:03:56,282 DEBUG [M:0;jenkins-hbase4:46229] cleaner.HFileCleaner(317): Stopping file delete threads 2023-09-04 06:03:56,283 INFO [M:0;jenkins-hbase4:46229] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-09-04 06:03:56,283 INFO [M:0;jenkins-hbase4:46229] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-09-04 06:03:56,283 INFO [M:0;jenkins-hbase4:46229] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-09-04 06:03:56,283 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-09-04 06:03:56,283 DEBUG [M:0;jenkins-hbase4:46229] master.HMaster(1512): Stopping service threads 2023-09-04 06:03:56,283 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:03:56,284 INFO [M:0;jenkins-hbase4:46229] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-09-04 06:03:56,284 ERROR [M:0;jenkins-hbase4:46229] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-09-04 06:03:56,284 INFO [M:0;jenkins-hbase4:46229] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-09-04 06:03:56,284 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 06:03:56,284 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-09-04 06:03:56,284 DEBUG [M:0;jenkins-hbase4:46229] zookeeper.ZKUtil(398): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-09-04 06:03:56,284 WARN [M:0;jenkins-hbase4:46229] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-09-04 06:03:56,285 INFO [M:0;jenkins-hbase4:46229] assignment.AssignmentManager(315): Stopping assignment manager 2023-09-04 06:03:56,285 INFO [M:0;jenkins-hbase4:46229] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-09-04 06:03:56,285 DEBUG [M:0;jenkins-hbase4:46229] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 06:03:56,285 INFO [M:0;jenkins-hbase4:46229] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:03:56,285 DEBUG [M:0;jenkins-hbase4:46229] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:03:56,285 DEBUG [M:0;jenkins-hbase4:46229] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 06:03:56,285 DEBUG [M:0;jenkins-hbase4:46229] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:03:56,285 INFO [M:0;jenkins-hbase4:46229] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=64.71 KB heapSize=78.42 KB 2023-09-04 06:03:56,297 INFO [M:0;jenkins-hbase4:46229] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=64.71 KB at sequenceid=160 (bloomFilter=true), to=hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/baab54f0307b4d388adc9a7b857f0bc7 2023-09-04 06:03:56,301 INFO [M:0;jenkins-hbase4:46229] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for baab54f0307b4d388adc9a7b857f0bc7 2023-09-04 06:03:56,302 DEBUG [M:0;jenkins-hbase4:46229] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/baab54f0307b4d388adc9a7b857f0bc7 as hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/baab54f0307b4d388adc9a7b857f0bc7 2023-09-04 06:03:56,306 INFO [M:0;jenkins-hbase4:46229] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for baab54f0307b4d388adc9a7b857f0bc7 2023-09-04 06:03:56,307 INFO [M:0;jenkins-hbase4:46229] regionserver.HStore(1080): Added hdfs://localhost:38163/user/jenkins/test-data/d3838801-ad32-1cc6-896f-3136e6eaa058/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/baab54f0307b4d388adc9a7b857f0bc7, entries=18, sequenceid=160, filesize=6.9 K 2023-09-04 06:03:56,307 INFO [M:0;jenkins-hbase4:46229] regionserver.HRegion(2948): Finished flush of dataSize ~64.71 KB/66268, heapSize ~78.41 KB/80288, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 22ms, sequenceid=160, compaction requested=false 2023-09-04 06:03:56,309 INFO [M:0;jenkins-hbase4:46229] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:03:56,309 DEBUG [M:0;jenkins-hbase4:46229] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 06:03:56,311 INFO [M:0;jenkins-hbase4:46229] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-09-04 06:03:56,311 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 06:03:56,312 INFO [M:0;jenkins-hbase4:46229] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46229 2023-09-04 06:03:56,314 DEBUG [M:0;jenkins-hbase4:46229] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,46229,1693807287459 already deleted, retry=false 2023-09-04 06:03:56,380 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): regionserver:44413-0x100758754360001, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:03:56,380 INFO [RS:0;jenkins-hbase4:44413] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,44413,1693807287501; zookeeper connection closed. 2023-09-04 06:03:56,380 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): regionserver:44413-0x100758754360001, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:03:56,381 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@31a14376] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@31a14376 2023-09-04 06:03:56,381 INFO [Listener at localhost/37417] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-09-04 06:03:56,480 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:03:56,480 INFO [M:0;jenkins-hbase4:46229] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46229,1693807287459; zookeeper connection closed. 2023-09-04 06:03:56,480 DEBUG [Listener at localhost/37417-EventThread] zookeeper.ZKWatcher(600): master:46229-0x100758754360000, quorum=127.0.0.1:56951, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:03:56,481 WARN [Listener at localhost/37417] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 06:03:56,484 INFO [Listener at localhost/37417] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:03:56,589 WARN [BP-824427623-172.31.14.131-1693807286898 heartbeating to localhost/127.0.0.1:38163] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 06:03:56,590 WARN [BP-824427623-172.31.14.131-1693807286898 heartbeating to localhost/127.0.0.1:38163] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-824427623-172.31.14.131-1693807286898 (Datanode Uuid f79419e4-9101-4ccb-92e1-69994ef4a42e) service to localhost/127.0.0.1:38163 2023-09-04 06:03:56,590 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/cluster_200fa038-4496-7b8d-6264-2148931f885c/dfs/data/data3/current/BP-824427623-172.31.14.131-1693807286898] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:03:56,590 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/cluster_200fa038-4496-7b8d-6264-2148931f885c/dfs/data/data4/current/BP-824427623-172.31.14.131-1693807286898] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:03:56,592 WARN [Listener at localhost/37417] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 06:03:56,595 INFO [Listener at localhost/37417] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:03:56,699 WARN [BP-824427623-172.31.14.131-1693807286898 heartbeating to localhost/127.0.0.1:38163] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 06:03:56,699 WARN [BP-824427623-172.31.14.131-1693807286898 heartbeating to localhost/127.0.0.1:38163] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-824427623-172.31.14.131-1693807286898 (Datanode Uuid 14656170-88c6-4379-82a4-7527f7d4d25f) service to localhost/127.0.0.1:38163 2023-09-04 06:03:56,700 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/cluster_200fa038-4496-7b8d-6264-2148931f885c/dfs/data/data1/current/BP-824427623-172.31.14.131-1693807286898] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:03:56,700 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/cluster_200fa038-4496-7b8d-6264-2148931f885c/dfs/data/data2/current/BP-824427623-172.31.14.131-1693807286898] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:03:56,711 INFO [Listener at localhost/37417] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:03:56,827 INFO [Listener at localhost/37417] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-09-04 06:03:56,860 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-09-04 06:03:56,869 INFO [Listener at localhost/37417] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=76 (was 64) - Thread LEAK? -, OpenFileDescriptor=466 (was 440) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=160 (was 71) - SystemLoadAverage LEAK? -, ProcessCount=164 (was 166), AvailableMemoryMB=9179 (was 8258) - AvailableMemoryMB LEAK? - 2023-09-04 06:03:56,878 INFO [Listener at localhost/37417] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=77, OpenFileDescriptor=466, MaxFileDescriptor=60000, SystemLoadAverage=160, ProcessCount=164, AvailableMemoryMB=9179 2023-09-04 06:03:56,878 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-09-04 06:03:56,878 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/hadoop.log.dir so I do NOT create it in target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f 2023-09-04 06:03:56,878 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/436119d8-ce25-0c69-bb74-053b21d91e83/hadoop.tmp.dir so I do NOT create it in target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f 2023-09-04 06:03:56,878 INFO [Listener at localhost/37417] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/cluster_84f50541-547e-f103-32a1-70596c6f7281, deleteOnExit=true 2023-09-04 06:03:56,878 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-09-04 06:03:56,878 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/test.cache.data in system properties and HBase conf 2023-09-04 06:03:56,878 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/hadoop.tmp.dir in system properties and HBase conf 2023-09-04 06:03:56,879 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/hadoop.log.dir in system properties and HBase conf 2023-09-04 06:03:56,879 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/mapreduce.cluster.local.dir in system properties and HBase conf 2023-09-04 06:03:56,879 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-09-04 06:03:56,879 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-09-04 06:03:56,879 DEBUG [Listener at localhost/37417] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-09-04 06:03:56,879 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-09-04 06:03:56,879 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-09-04 06:03:56,879 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-09-04 06:03:56,879 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 06:03:56,880 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-09-04 06:03:56,880 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-09-04 06:03:56,880 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 06:03:56,880 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 06:03:56,880 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-09-04 06:03:56,880 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/nfs.dump.dir in system properties and HBase conf 2023-09-04 06:03:56,880 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/java.io.tmpdir in system properties and HBase conf 2023-09-04 06:03:56,880 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 06:03:56,880 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-09-04 06:03:56,880 INFO [Listener at localhost/37417] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-09-04 06:03:56,884 WARN [Listener at localhost/37417] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 06:03:56,885 WARN [Listener at localhost/37417] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 06:03:56,924 WARN [Listener at localhost/37417] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 06:03:56,925 INFO [Listener at localhost/37417] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 06:03:56,930 INFO [Listener at localhost/37417] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/java.io.tmpdir/Jetty_localhost_34697_hdfs____.3pb5hf/webapp 2023-09-04 06:03:57,021 INFO [Listener at localhost/37417] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34697 2023-09-04 06:03:57,025 WARN [Listener at localhost/37417] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 06:03:57,025 WARN [Listener at localhost/37417] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 06:03:57,070 WARN [Listener at localhost/38345] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 06:03:57,080 WARN [Listener at localhost/38345] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 06:03:57,083 WARN [Listener at localhost/38345] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 06:03:57,084 INFO [Listener at localhost/38345] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 06:03:57,088 INFO [Listener at localhost/38345] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/java.io.tmpdir/Jetty_localhost_37927_datanode____ybphdt/webapp 2023-09-04 06:03:57,180 INFO [Listener at localhost/38345] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37927 2023-09-04 06:03:57,187 WARN [Listener at localhost/36817] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 06:03:57,199 WARN [Listener at localhost/36817] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 06:03:57,201 WARN [Listener at localhost/36817] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 06:03:57,203 INFO [Listener at localhost/36817] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 06:03:57,213 INFO [Listener at localhost/36817] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/java.io.tmpdir/Jetty_localhost_35793_datanode____x9dw0m/webapp 2023-09-04 06:03:57,286 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x1d85d997ab4e0cb7: Processing first storage report for DS-19f16f38-f7cb-4aed-9e1e-b7ef9ca89d28 from datanode 4d990609-f9da-404b-abf3-715efcc4d246 2023-09-04 06:03:57,286 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x1d85d997ab4e0cb7: from storage DS-19f16f38-f7cb-4aed-9e1e-b7ef9ca89d28 node DatanodeRegistration(127.0.0.1:44193, datanodeUuid=4d990609-f9da-404b-abf3-715efcc4d246, infoPort=41969, infoSecurePort=0, ipcPort=36817, storageInfo=lv=-57;cid=testClusterID;nsid=599791442;c=1693807436887), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:03:57,286 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x1d85d997ab4e0cb7: Processing first storage report for DS-37edaf10-11b8-4f66-89b4-d18240ca5080 from datanode 4d990609-f9da-404b-abf3-715efcc4d246 2023-09-04 06:03:57,286 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x1d85d997ab4e0cb7: from storage DS-37edaf10-11b8-4f66-89b4-d18240ca5080 node DatanodeRegistration(127.0.0.1:44193, datanodeUuid=4d990609-f9da-404b-abf3-715efcc4d246, infoPort=41969, infoSecurePort=0, ipcPort=36817, storageInfo=lv=-57;cid=testClusterID;nsid=599791442;c=1693807436887), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:03:57,310 INFO [Listener at localhost/36817] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35793 2023-09-04 06:03:57,317 WARN [Listener at localhost/44821] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 06:03:57,404 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xdc05defa0aca0f21: Processing first storage report for DS-2107ec5c-c85c-4daf-88f7-f47f0e008713 from datanode 34fed369-8aa5-4aef-a3ae-81265950af3f 2023-09-04 06:03:57,404 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xdc05defa0aca0f21: from storage DS-2107ec5c-c85c-4daf-88f7-f47f0e008713 node DatanodeRegistration(127.0.0.1:44035, datanodeUuid=34fed369-8aa5-4aef-a3ae-81265950af3f, infoPort=38569, infoSecurePort=0, ipcPort=44821, storageInfo=lv=-57;cid=testClusterID;nsid=599791442;c=1693807436887), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:03:57,404 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xdc05defa0aca0f21: Processing first storage report for DS-445c1783-7aa1-40a5-a06a-83c2ed9ea44c from datanode 34fed369-8aa5-4aef-a3ae-81265950af3f 2023-09-04 06:03:57,404 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xdc05defa0aca0f21: from storage DS-445c1783-7aa1-40a5-a06a-83c2ed9ea44c node DatanodeRegistration(127.0.0.1:44035, datanodeUuid=34fed369-8aa5-4aef-a3ae-81265950af3f, infoPort=38569, infoSecurePort=0, ipcPort=44821, storageInfo=lv=-57;cid=testClusterID;nsid=599791442;c=1693807436887), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 06:03:57,424 DEBUG [Listener at localhost/44821] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f 2023-09-04 06:03:57,426 INFO [Listener at localhost/44821] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/cluster_84f50541-547e-f103-32a1-70596c6f7281/zookeeper_0, clientPort=54525, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/cluster_84f50541-547e-f103-32a1-70596c6f7281/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/cluster_84f50541-547e-f103-32a1-70596c6f7281/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-09-04 06:03:57,427 INFO [Listener at localhost/44821] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=54525 2023-09-04 06:03:57,427 INFO [Listener at localhost/44821] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:03:57,428 INFO [Listener at localhost/44821] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:03:57,441 INFO [Listener at localhost/44821] util.FSUtils(471): Created version file at hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef with version=8 2023-09-04 06:03:57,442 INFO [Listener at localhost/44821] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:42011/user/jenkins/test-data/46400cd7-e274-ee1b-608d-098ae8bed914/hbase-staging 2023-09-04 06:03:57,444 INFO [Listener at localhost/44821] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 06:03:57,444 INFO [Listener at localhost/44821] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:03:57,444 INFO [Listener at localhost/44821] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 06:03:57,444 INFO [Listener at localhost/44821] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 06:03:57,444 INFO [Listener at localhost/44821] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:03:57,445 INFO [Listener at localhost/44821] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 06:03:57,445 INFO [Listener at localhost/44821] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 06:03:57,446 INFO [Listener at localhost/44821] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40313 2023-09-04 06:03:57,447 INFO [Listener at localhost/44821] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:03:57,448 INFO [Listener at localhost/44821] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:03:57,449 INFO [Listener at localhost/44821] zookeeper.RecoverableZooKeeper(93): Process identifier=master:40313 connecting to ZooKeeper ensemble=127.0.0.1:54525 2023-09-04 06:03:57,456 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:403130x0, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 06:03:57,457 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:40313-0x10075899e180000 connected 2023-09-04 06:03:57,470 DEBUG [Listener at localhost/44821] zookeeper.ZKUtil(164): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 06:03:57,471 DEBUG [Listener at localhost/44821] zookeeper.ZKUtil(164): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:03:57,471 DEBUG [Listener at localhost/44821] zookeeper.ZKUtil(164): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 06:03:57,472 DEBUG [Listener at localhost/44821] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40313 2023-09-04 06:03:57,472 DEBUG [Listener at localhost/44821] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40313 2023-09-04 06:03:57,472 DEBUG [Listener at localhost/44821] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40313 2023-09-04 06:03:57,472 DEBUG [Listener at localhost/44821] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40313 2023-09-04 06:03:57,472 DEBUG [Listener at localhost/44821] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40313 2023-09-04 06:03:57,473 INFO [Listener at localhost/44821] master.HMaster(444): hbase.rootdir=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef, hbase.cluster.distributed=false 2023-09-04 06:03:57,486 INFO [Listener at localhost/44821] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 06:03:57,486 INFO [Listener at localhost/44821] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:03:57,486 INFO [Listener at localhost/44821] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 06:03:57,486 INFO [Listener at localhost/44821] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 06:03:57,486 INFO [Listener at localhost/44821] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 06:03:57,486 INFO [Listener at localhost/44821] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 06:03:57,486 INFO [Listener at localhost/44821] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 06:03:57,488 INFO [Listener at localhost/44821] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35067 2023-09-04 06:03:57,489 INFO [Listener at localhost/44821] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-04 06:03:57,490 DEBUG [Listener at localhost/44821] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-09-04 06:03:57,490 INFO [Listener at localhost/44821] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:03:57,491 INFO [Listener at localhost/44821] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:03:57,492 INFO [Listener at localhost/44821] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:35067 connecting to ZooKeeper ensemble=127.0.0.1:54525 2023-09-04 06:03:57,495 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): regionserver:350670x0, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 06:03:57,497 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:35067-0x10075899e180001 connected 2023-09-04 06:03:57,497 DEBUG [Listener at localhost/44821] zookeeper.ZKUtil(164): regionserver:35067-0x10075899e180001, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 06:03:57,497 DEBUG [Listener at localhost/44821] zookeeper.ZKUtil(164): regionserver:35067-0x10075899e180001, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:03:57,498 DEBUG [Listener at localhost/44821] zookeeper.ZKUtil(164): regionserver:35067-0x10075899e180001, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 06:03:57,499 DEBUG [Listener at localhost/44821] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35067 2023-09-04 06:03:57,499 DEBUG [Listener at localhost/44821] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35067 2023-09-04 06:03:57,499 DEBUG [Listener at localhost/44821] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35067 2023-09-04 06:03:57,499 DEBUG [Listener at localhost/44821] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35067 2023-09-04 06:03:57,499 DEBUG [Listener at localhost/44821] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35067 2023-09-04 06:03:57,501 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,40313,1693807437443 2023-09-04 06:03:57,504 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 06:03:57,504 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,40313,1693807437443 2023-09-04 06:03:57,505 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 06:03:57,505 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): regionserver:35067-0x10075899e180001, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 06:03:57,505 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:03:57,506 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 06:03:57,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,40313,1693807437443 from backup master directory 2023-09-04 06:03:57,507 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 06:03:57,508 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,40313,1693807437443 2023-09-04 06:03:57,508 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 06:03:57,508 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 06:03:57,508 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,40313,1693807437443 2023-09-04 06:03:57,521 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/hbase.id with ID: bd385f7c-fbb3-4a98-bdd1-01b03d9fd349 2023-09-04 06:03:57,533 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:03:57,536 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:03:57,544 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x2ce65949 to 127.0.0.1:54525 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 06:03:57,549 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c759d66, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 06:03:57,549 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-04 06:03:57,549 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-09-04 06:03:57,550 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 06:03:57,551 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/MasterData/data/master/store-tmp 2023-09-04 06:03:57,559 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:03:57,559 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 06:03:57,559 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:03:57,559 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:03:57,559 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 06:03:57,559 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:03:57,559 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:03:57,559 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 06:03:57,560 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/MasterData/WALs/jenkins-hbase4.apache.org,40313,1693807437443 2023-09-04 06:03:57,562 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40313%2C1693807437443, suffix=, logDir=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/MasterData/WALs/jenkins-hbase4.apache.org,40313,1693807437443, archiveDir=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/MasterData/oldWALs, maxLogs=10 2023-09-04 06:03:57,577 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44035,DS-2107ec5c-c85c-4daf-88f7-f47f0e008713,DISK] 2023-09-04 06:03:57,577 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44193,DS-19f16f38-f7cb-4aed-9e1e-b7ef9ca89d28,DISK] 2023-09-04 06:03:57,580 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/MasterData/WALs/jenkins-hbase4.apache.org,40313,1693807437443/jenkins-hbase4.apache.org%2C40313%2C1693807437443.1693807437562 2023-09-04 06:03:57,580 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44035,DS-2107ec5c-c85c-4daf-88f7-f47f0e008713,DISK], DatanodeInfoWithStorage[127.0.0.1:44193,DS-19f16f38-f7cb-4aed-9e1e-b7ef9ca89d28,DISK]] 2023-09-04 06:03:57,580 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-09-04 06:03:57,580 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:03:57,580 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:03:57,580 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:03:57,582 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:03:57,583 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-09-04 06:03:57,584 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-09-04 06:03:57,584 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:03:57,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:03:57,586 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:03:57,588 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 06:03:57,590 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 06:03:57,590 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=720122, jitterRate=-0.08431781828403473}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 06:03:57,590 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 06:03:57,591 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-09-04 06:03:57,592 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-09-04 06:03:57,592 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-09-04 06:03:57,592 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-09-04 06:03:57,592 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-09-04 06:03:57,592 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 0 msec 2023-09-04 06:03:57,592 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-09-04 06:03:57,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-09-04 06:03:57,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-09-04 06:03:57,605 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-09-04 06:03:57,605 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-09-04 06:03:57,605 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-09-04 06:03:57,605 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-09-04 06:03:57,606 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-09-04 06:03:57,608 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:03:57,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-09-04 06:03:57,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-09-04 06:03:57,610 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-09-04 06:03:57,611 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): regionserver:35067-0x10075899e180001, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 06:03:57,611 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 06:03:57,611 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:03:57,611 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,40313,1693807437443, sessionid=0x10075899e180000, setting cluster-up flag (Was=false) 2023-09-04 06:03:57,616 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:03:57,621 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-09-04 06:03:57,622 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40313,1693807437443 2023-09-04 06:03:57,626 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:03:57,632 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-09-04 06:03:57,632 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40313,1693807437443 2023-09-04 06:03:57,633 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/.hbase-snapshot/.tmp 2023-09-04 06:03:57,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-09-04 06:03:57,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:03:57,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:03:57,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:03:57,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 06:03:57,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-09-04 06:03:57,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:03:57,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 06:03:57,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:03:57,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693807467637 2023-09-04 06:03:57,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-09-04 06:03:57,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-09-04 06:03:57,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-09-04 06:03:57,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-09-04 06:03:57,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-09-04 06:03:57,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-09-04 06:03:57,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 06:03:57,637 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 06:03:57,638 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-09-04 06:03:57,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-09-04 06:03:57,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-09-04 06:03:57,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-09-04 06:03:57,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-09-04 06:03:57,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-09-04 06:03:57,638 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693807437638,5,FailOnTimeoutGroup] 2023-09-04 06:03:57,639 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693807437639,5,FailOnTimeoutGroup] 2023-09-04 06:03:57,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 06:03:57,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-09-04 06:03:57,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-09-04 06:03:57,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-09-04 06:03:57,639 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 06:03:57,648 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 06:03:57,649 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 06:03:57,649 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef 2023-09-04 06:03:57,656 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:03:57,657 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 06:03:57,659 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/info 2023-09-04 06:03:57,659 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 06:03:57,660 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:03:57,660 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 06:03:57,661 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/rep_barrier 2023-09-04 06:03:57,661 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 06:03:57,662 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:03:57,662 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 06:03:57,663 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/table 2023-09-04 06:03:57,663 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 06:03:57,663 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:03:57,664 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740 2023-09-04 06:03:57,664 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740 2023-09-04 06:03:57,666 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 06:03:57,667 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 06:03:57,669 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 06:03:57,670 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=880223, jitterRate=0.11926265060901642}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 06:03:57,670 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 06:03:57,670 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 06:03:57,670 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 06:03:57,670 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 06:03:57,670 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 06:03:57,670 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 06:03:57,670 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 06:03:57,670 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 06:03:57,671 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 06:03:57,671 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-09-04 06:03:57,671 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-09-04 06:03:57,672 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-09-04 06:03:57,673 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-09-04 06:03:57,701 INFO [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(951): ClusterId : bd385f7c-fbb3-4a98-bdd1-01b03d9fd349 2023-09-04 06:03:57,702 DEBUG [RS:0;jenkins-hbase4:35067] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-09-04 06:03:57,705 DEBUG [RS:0;jenkins-hbase4:35067] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-09-04 06:03:57,705 DEBUG [RS:0;jenkins-hbase4:35067] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-09-04 06:03:57,707 DEBUG [RS:0;jenkins-hbase4:35067] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-09-04 06:03:57,708 DEBUG [RS:0;jenkins-hbase4:35067] zookeeper.ReadOnlyZKClient(139): Connect 0x279d72ae to 127.0.0.1:54525 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 06:03:57,711 DEBUG [RS:0;jenkins-hbase4:35067] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@295a882d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 06:03:57,712 DEBUG [RS:0;jenkins-hbase4:35067] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d6519de, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 06:03:57,720 DEBUG [RS:0;jenkins-hbase4:35067] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:35067 2023-09-04 06:03:57,720 INFO [RS:0;jenkins-hbase4:35067] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-09-04 06:03:57,720 INFO [RS:0;jenkins-hbase4:35067] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-09-04 06:03:57,720 DEBUG [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(1022): About to register with Master. 2023-09-04 06:03:57,721 INFO [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,40313,1693807437443 with isa=jenkins-hbase4.apache.org/172.31.14.131:35067, startcode=1693807437485 2023-09-04 06:03:57,721 DEBUG [RS:0;jenkins-hbase4:35067] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-09-04 06:03:57,724 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38701, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-09-04 06:03:57,725 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40313] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,35067,1693807437485 2023-09-04 06:03:57,725 DEBUG [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef 2023-09-04 06:03:57,725 DEBUG [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:38345 2023-09-04 06:03:57,725 DEBUG [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-09-04 06:03:57,728 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 06:03:57,728 DEBUG [RS:0;jenkins-hbase4:35067] zookeeper.ZKUtil(162): regionserver:35067-0x10075899e180001, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35067,1693807437485 2023-09-04 06:03:57,729 WARN [RS:0;jenkins-hbase4:35067] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 06:03:57,729 INFO [RS:0;jenkins-hbase4:35067] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 06:03:57,729 DEBUG [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(1948): logDir=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/WALs/jenkins-hbase4.apache.org,35067,1693807437485 2023-09-04 06:03:57,729 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,35067,1693807437485] 2023-09-04 06:03:57,732 DEBUG [RS:0;jenkins-hbase4:35067] zookeeper.ZKUtil(162): regionserver:35067-0x10075899e180001, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35067,1693807437485 2023-09-04 06:03:57,733 DEBUG [RS:0;jenkins-hbase4:35067] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-09-04 06:03:57,733 INFO [RS:0;jenkins-hbase4:35067] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-09-04 06:03:57,734 INFO [RS:0;jenkins-hbase4:35067] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-09-04 06:03:57,734 INFO [RS:0;jenkins-hbase4:35067] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-09-04 06:03:57,735 INFO [RS:0;jenkins-hbase4:35067] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 06:03:57,735 INFO [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-09-04 06:03:57,736 INFO [RS:0;jenkins-hbase4:35067] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-09-04 06:03:57,736 DEBUG [RS:0;jenkins-hbase4:35067] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:03:57,736 DEBUG [RS:0;jenkins-hbase4:35067] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:03:57,736 DEBUG [RS:0;jenkins-hbase4:35067] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:03:57,736 DEBUG [RS:0;jenkins-hbase4:35067] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:03:57,736 DEBUG [RS:0;jenkins-hbase4:35067] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:03:57,736 DEBUG [RS:0;jenkins-hbase4:35067] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 06:03:57,736 DEBUG [RS:0;jenkins-hbase4:35067] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:03:57,736 DEBUG [RS:0;jenkins-hbase4:35067] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:03:57,736 DEBUG [RS:0;jenkins-hbase4:35067] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:03:57,736 DEBUG [RS:0;jenkins-hbase4:35067] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 06:03:57,737 INFO [RS:0;jenkins-hbase4:35067] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 06:03:57,737 INFO [RS:0;jenkins-hbase4:35067] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 06:03:57,737 INFO [RS:0;jenkins-hbase4:35067] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-09-04 06:03:57,752 INFO [RS:0;jenkins-hbase4:35067] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-09-04 06:03:57,752 INFO [RS:0;jenkins-hbase4:35067] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35067,1693807437485-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 06:03:57,758 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-09-04 06:03:57,763 INFO [RS:0;jenkins-hbase4:35067] regionserver.Replication(203): jenkins-hbase4.apache.org,35067,1693807437485 started 2023-09-04 06:03:57,763 INFO [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,35067,1693807437485, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:35067, sessionid=0x10075899e180001 2023-09-04 06:03:57,763 DEBUG [RS:0;jenkins-hbase4:35067] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-09-04 06:03:57,763 DEBUG [RS:0;jenkins-hbase4:35067] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,35067,1693807437485 2023-09-04 06:03:57,763 DEBUG [RS:0;jenkins-hbase4:35067] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35067,1693807437485' 2023-09-04 06:03:57,763 DEBUG [RS:0;jenkins-hbase4:35067] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 06:03:57,764 DEBUG [RS:0;jenkins-hbase4:35067] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 06:03:57,764 DEBUG [RS:0;jenkins-hbase4:35067] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-09-04 06:03:57,764 DEBUG [RS:0;jenkins-hbase4:35067] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-09-04 06:03:57,764 DEBUG [RS:0;jenkins-hbase4:35067] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,35067,1693807437485 2023-09-04 06:03:57,764 DEBUG [RS:0;jenkins-hbase4:35067] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35067,1693807437485' 2023-09-04 06:03:57,764 DEBUG [RS:0;jenkins-hbase4:35067] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-09-04 06:03:57,764 DEBUG [RS:0;jenkins-hbase4:35067] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-09-04 06:03:57,765 DEBUG [RS:0;jenkins-hbase4:35067] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-09-04 06:03:57,765 INFO [RS:0;jenkins-hbase4:35067] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-09-04 06:03:57,765 INFO [RS:0;jenkins-hbase4:35067] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-09-04 06:03:57,823 DEBUG [jenkins-hbase4:40313] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-09-04 06:03:57,824 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,35067,1693807437485, state=OPENING 2023-09-04 06:03:57,826 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-09-04 06:03:57,827 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:03:57,828 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 06:03:57,828 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,35067,1693807437485}] 2023-09-04 06:03:57,867 INFO [RS:0;jenkins-hbase4:35067] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35067%2C1693807437485, suffix=, logDir=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/WALs/jenkins-hbase4.apache.org,35067,1693807437485, archiveDir=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/oldWALs, maxLogs=32 2023-09-04 06:03:57,885 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44035,DS-2107ec5c-c85c-4daf-88f7-f47f0e008713,DISK] 2023-09-04 06:03:57,885 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44193,DS-19f16f38-f7cb-4aed-9e1e-b7ef9ca89d28,DISK] 2023-09-04 06:03:57,887 INFO [RS:0;jenkins-hbase4:35067] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/WALs/jenkins-hbase4.apache.org,35067,1693807437485/jenkins-hbase4.apache.org%2C35067%2C1693807437485.1693807437867 2023-09-04 06:03:57,887 DEBUG [RS:0;jenkins-hbase4:35067] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44035,DS-2107ec5c-c85c-4daf-88f7-f47f0e008713,DISK], DatanodeInfoWithStorage[127.0.0.1:44193,DS-19f16f38-f7cb-4aed-9e1e-b7ef9ca89d28,DISK]] 2023-09-04 06:03:57,981 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,35067,1693807437485 2023-09-04 06:03:57,982 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-09-04 06:03:57,984 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45660, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-09-04 06:03:57,987 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-09-04 06:03:57,987 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 06:03:57,989 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35067%2C1693807437485.meta, suffix=.meta, logDir=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/WALs/jenkins-hbase4.apache.org,35067,1693807437485, archiveDir=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/oldWALs, maxLogs=32 2023-09-04 06:03:58,005 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44035,DS-2107ec5c-c85c-4daf-88f7-f47f0e008713,DISK] 2023-09-04 06:03:58,005 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44193,DS-19f16f38-f7cb-4aed-9e1e-b7ef9ca89d28,DISK] 2023-09-04 06:03:58,007 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/WALs/jenkins-hbase4.apache.org,35067,1693807437485/jenkins-hbase4.apache.org%2C35067%2C1693807437485.meta.1693807437989.meta 2023-09-04 06:03:58,007 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44035,DS-2107ec5c-c85c-4daf-88f7-f47f0e008713,DISK], DatanodeInfoWithStorage[127.0.0.1:44193,DS-19f16f38-f7cb-4aed-9e1e-b7ef9ca89d28,DISK]] 2023-09-04 06:03:58,007 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-09-04 06:03:58,007 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-09-04 06:03:58,007 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-09-04 06:03:58,008 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-09-04 06:03:58,008 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-09-04 06:03:58,008 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:03:58,008 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-09-04 06:03:58,008 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-09-04 06:03:58,009 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 06:03:58,010 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/info 2023-09-04 06:03:58,010 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/info 2023-09-04 06:03:58,010 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 06:03:58,011 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:03:58,011 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 06:03:58,012 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/rep_barrier 2023-09-04 06:03:58,012 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/rep_barrier 2023-09-04 06:03:58,012 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 06:03:58,013 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:03:58,013 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 06:03:58,014 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/table 2023-09-04 06:03:58,014 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/table 2023-09-04 06:03:58,014 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 06:03:58,014 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:03:58,015 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740 2023-09-04 06:03:58,016 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740 2023-09-04 06:03:58,018 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 06:03:58,019 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 06:03:58,020 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=691719, jitterRate=-0.12043449282646179}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 06:03:58,020 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 06:03:58,021 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693807437981 2023-09-04 06:03:58,025 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-09-04 06:03:58,025 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-09-04 06:03:58,026 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,35067,1693807437485, state=OPEN 2023-09-04 06:03:58,027 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-09-04 06:03:58,028 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 06:03:58,029 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-09-04 06:03:58,029 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,35067,1693807437485 in 199 msec 2023-09-04 06:03:58,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-09-04 06:03:58,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 358 msec 2023-09-04 06:03:58,032 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 397 msec 2023-09-04 06:03:58,032 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693807438032, completionTime=-1 2023-09-04 06:03:58,032 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-09-04 06:03:58,032 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-09-04 06:03:58,034 DEBUG [hconnection-0x77716e04-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 06:03:58,036 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45666, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 06:03:58,037 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-09-04 06:03:58,037 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693807498037 2023-09-04 06:03:58,037 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693807558037 2023-09-04 06:03:58,037 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-09-04 06:03:58,044 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40313,1693807437443-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 06:03:58,044 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40313,1693807437443-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 06:03:58,044 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40313,1693807437443-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 06:03:58,044 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:40313, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 06:03:58,044 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-09-04 06:03:58,044 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-09-04 06:03:58,044 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 06:03:58,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-09-04 06:03:58,045 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-09-04 06:03:58,046 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-09-04 06:03:58,047 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-04 06:03:58,049 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/.tmp/data/hbase/namespace/b543154e41d822c029ea4a240097fa55 2023-09-04 06:03:58,049 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/.tmp/data/hbase/namespace/b543154e41d822c029ea4a240097fa55 empty. 2023-09-04 06:03:58,050 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/.tmp/data/hbase/namespace/b543154e41d822c029ea4a240097fa55 2023-09-04 06:03:58,050 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-09-04 06:03:58,061 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-09-04 06:03:58,062 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => b543154e41d822c029ea4a240097fa55, NAME => 'hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/.tmp 2023-09-04 06:03:58,071 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:03:58,072 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing b543154e41d822c029ea4a240097fa55, disabling compactions & flushes 2023-09-04 06:03:58,072 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55. 2023-09-04 06:03:58,072 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55. 2023-09-04 06:03:58,072 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55. after waiting 0 ms 2023-09-04 06:03:58,072 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55. 2023-09-04 06:03:58,072 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55. 2023-09-04 06:03:58,072 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for b543154e41d822c029ea4a240097fa55: 2023-09-04 06:03:58,074 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-09-04 06:03:58,075 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693807438075"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693807438075"}]},"ts":"1693807438075"} 2023-09-04 06:03:58,077 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-04 06:03:58,078 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-04 06:03:58,078 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693807438078"}]},"ts":"1693807438078"} 2023-09-04 06:03:58,079 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-09-04 06:03:58,091 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=b543154e41d822c029ea4a240097fa55, ASSIGN}] 2023-09-04 06:03:58,092 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=b543154e41d822c029ea4a240097fa55, ASSIGN 2023-09-04 06:03:58,093 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=b543154e41d822c029ea4a240097fa55, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35067,1693807437485; forceNewPlan=false, retain=false 2023-09-04 06:03:58,244 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=b543154e41d822c029ea4a240097fa55, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35067,1693807437485 2023-09-04 06:03:58,244 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693807438244"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693807438244"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693807438244"}]},"ts":"1693807438244"} 2023-09-04 06:03:58,246 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure b543154e41d822c029ea4a240097fa55, server=jenkins-hbase4.apache.org,35067,1693807437485}] 2023-09-04 06:03:58,401 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55. 2023-09-04 06:03:58,402 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b543154e41d822c029ea4a240097fa55, NAME => 'hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55.', STARTKEY => '', ENDKEY => ''} 2023-09-04 06:03:58,402 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace b543154e41d822c029ea4a240097fa55 2023-09-04 06:03:58,402 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 06:03:58,402 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b543154e41d822c029ea4a240097fa55 2023-09-04 06:03:58,402 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b543154e41d822c029ea4a240097fa55 2023-09-04 06:03:58,403 INFO [StoreOpener-b543154e41d822c029ea4a240097fa55-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region b543154e41d822c029ea4a240097fa55 2023-09-04 06:03:58,405 DEBUG [StoreOpener-b543154e41d822c029ea4a240097fa55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/namespace/b543154e41d822c029ea4a240097fa55/info 2023-09-04 06:03:58,405 DEBUG [StoreOpener-b543154e41d822c029ea4a240097fa55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/namespace/b543154e41d822c029ea4a240097fa55/info 2023-09-04 06:03:58,405 INFO [StoreOpener-b543154e41d822c029ea4a240097fa55-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b543154e41d822c029ea4a240097fa55 columnFamilyName info 2023-09-04 06:03:58,406 INFO [StoreOpener-b543154e41d822c029ea4a240097fa55-1] regionserver.HStore(310): Store=b543154e41d822c029ea4a240097fa55/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 06:03:58,406 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/namespace/b543154e41d822c029ea4a240097fa55 2023-09-04 06:03:58,407 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/namespace/b543154e41d822c029ea4a240097fa55 2023-09-04 06:03:58,410 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b543154e41d822c029ea4a240097fa55 2023-09-04 06:03:58,412 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/namespace/b543154e41d822c029ea4a240097fa55/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 06:03:58,412 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b543154e41d822c029ea4a240097fa55; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=755058, jitterRate=-0.039894238114356995}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 06:03:58,412 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b543154e41d822c029ea4a240097fa55: 2023-09-04 06:03:58,413 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55., pid=6, masterSystemTime=1693807438398 2023-09-04 06:03:58,414 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55. 2023-09-04 06:03:58,414 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55. 2023-09-04 06:03:58,415 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=b543154e41d822c029ea4a240097fa55, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35067,1693807437485 2023-09-04 06:03:58,415 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693807438415"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693807438415"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693807438415"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693807438415"}]},"ts":"1693807438415"} 2023-09-04 06:03:58,418 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-09-04 06:03:58,418 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure b543154e41d822c029ea4a240097fa55, server=jenkins-hbase4.apache.org,35067,1693807437485 in 170 msec 2023-09-04 06:03:58,420 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-09-04 06:03:58,421 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=b543154e41d822c029ea4a240097fa55, ASSIGN in 328 msec 2023-09-04 06:03:58,421 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-04 06:03:58,421 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693807438421"}]},"ts":"1693807438421"} 2023-09-04 06:03:58,423 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-09-04 06:03:58,425 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-09-04 06:03:58,426 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 381 msec 2023-09-04 06:03:58,446 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-09-04 06:03:58,448 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-09-04 06:03:58,448 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:03:58,453 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-09-04 06:03:58,460 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 06:03:58,464 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 11 msec 2023-09-04 06:03:58,474 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-09-04 06:03:58,481 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 06:03:58,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 8 msec 2023-09-04 06:03:58,492 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-09-04 06:03:58,494 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-09-04 06:03:58,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 0.986sec 2023-09-04 06:03:58,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-09-04 06:03:58,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-09-04 06:03:58,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-09-04 06:03:58,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40313,1693807437443-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-09-04 06:03:58,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40313,1693807437443-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-09-04 06:03:58,496 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-09-04 06:03:58,502 DEBUG [Listener at localhost/44821] zookeeper.ReadOnlyZKClient(139): Connect 0x71198554 to 127.0.0.1:54525 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 06:03:58,508 DEBUG [Listener at localhost/44821] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7398dc3b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 06:03:58,510 DEBUG [hconnection-0x73c2eab4-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 06:03:58,511 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45682, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 06:03:58,513 INFO [Listener at localhost/44821] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,40313,1693807437443 2023-09-04 06:03:58,513 INFO [Listener at localhost/44821] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 06:03:58,519 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-09-04 06:03:58,519 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:03:58,520 INFO [Listener at localhost/44821] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-09-04 06:03:58,520 INFO [Listener at localhost/44821] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 06:03:58,522 INFO [Listener at localhost/44821] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=test.com%2C8080%2C1, suffix=, logDir=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/WALs/test.com,8080,1, archiveDir=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/oldWALs, maxLogs=32 2023-09-04 06:03:58,538 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44193,DS-19f16f38-f7cb-4aed-9e1e-b7ef9ca89d28,DISK] 2023-09-04 06:03:58,538 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44035,DS-2107ec5c-c85c-4daf-88f7-f47f0e008713,DISK] 2023-09-04 06:03:58,540 INFO [Listener at localhost/44821] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/WALs/test.com,8080,1/test.com%2C8080%2C1.1693807438522 2023-09-04 06:03:58,541 DEBUG [Listener at localhost/44821] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44193,DS-19f16f38-f7cb-4aed-9e1e-b7ef9ca89d28,DISK], DatanodeInfoWithStorage[127.0.0.1:44035,DS-2107ec5c-c85c-4daf-88f7-f47f0e008713,DISK]] 2023-09-04 06:03:58,555 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44193,DS-19f16f38-f7cb-4aed-9e1e-b7ef9ca89d28,DISK] 2023-09-04 06:03:58,555 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44035,DS-2107ec5c-c85c-4daf-88f7-f47f0e008713,DISK] 2023-09-04 06:03:58,556 INFO [Listener at localhost/44821] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/WALs/test.com,8080,1/test.com%2C8080%2C1.1693807438522 with entries=0, filesize=83 B; new WAL /user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/WALs/test.com,8080,1/test.com%2C8080%2C1.1693807438541 2023-09-04 06:03:58,559 DEBUG [Listener at localhost/44821] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44193,DS-19f16f38-f7cb-4aed-9e1e-b7ef9ca89d28,DISK], DatanodeInfoWithStorage[127.0.0.1:44035,DS-2107ec5c-c85c-4daf-88f7-f47f0e008713,DISK]] 2023-09-04 06:03:58,559 DEBUG [Listener at localhost/44821] wal.AbstractFSWAL(716): hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/WALs/test.com,8080,1/test.com%2C8080%2C1.1693807438522 is not closed yet, will try archiving it next time 2023-09-04 06:03:58,566 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/WALs/test.com,8080,1/test.com%2C8080%2C1.1693807438522 to hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/oldWALs/test.com%2C8080%2C1.1693807438522 2023-09-04 06:03:58,569 DEBUG [Listener at localhost/44821] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/oldWALs 2023-09-04 06:03:58,569 INFO [Listener at localhost/44821] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL test.com%2C8080%2C1:(num 1693807438541) 2023-09-04 06:03:58,569 INFO [Listener at localhost/44821] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-09-04 06:03:58,569 DEBUG [Listener at localhost/44821] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x71198554 to 127.0.0.1:54525 2023-09-04 06:03:58,569 DEBUG [Listener at localhost/44821] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:03:58,570 DEBUG [Listener at localhost/44821] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-09-04 06:03:58,570 DEBUG [Listener at localhost/44821] util.JVMClusterUtil(257): Found active master hash=574763372, stopped=false 2023-09-04 06:03:58,570 INFO [Listener at localhost/44821] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,40313,1693807437443 2023-09-04 06:03:58,572 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): regionserver:35067-0x10075899e180001, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 06:03:58,572 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 06:03:58,572 INFO [Listener at localhost/44821] procedure2.ProcedureExecutor(628): Stopping 2023-09-04 06:03:58,572 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:03:58,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35067-0x10075899e180001, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:03:58,573 DEBUG [Listener at localhost/44821] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2ce65949 to 127.0.0.1:54525 2023-09-04 06:03:58,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 06:03:58,573 DEBUG [Listener at localhost/44821] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:03:58,573 INFO [Listener at localhost/44821] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,35067,1693807437485' ***** 2023-09-04 06:03:58,574 INFO [Listener at localhost/44821] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-09-04 06:03:58,574 INFO [RS:0;jenkins-hbase4:35067] regionserver.HeapMemoryManager(220): Stopping 2023-09-04 06:03:58,574 INFO [RS:0;jenkins-hbase4:35067] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-09-04 06:03:58,574 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-09-04 06:03:58,574 INFO [RS:0;jenkins-hbase4:35067] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-09-04 06:03:58,575 INFO [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(3305): Received CLOSE for b543154e41d822c029ea4a240097fa55 2023-09-04 06:03:58,575 INFO [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,35067,1693807437485 2023-09-04 06:03:58,575 DEBUG [RS:0;jenkins-hbase4:35067] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x279d72ae to 127.0.0.1:54525 2023-09-04 06:03:58,575 DEBUG [RS:0;jenkins-hbase4:35067] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:03:58,575 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b543154e41d822c029ea4a240097fa55, disabling compactions & flushes 2023-09-04 06:03:58,575 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55. 2023-09-04 06:03:58,576 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55. 2023-09-04 06:03:58,576 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55. after waiting 0 ms 2023-09-04 06:03:58,576 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55. 2023-09-04 06:03:58,576 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing b543154e41d822c029ea4a240097fa55 1/1 column families, dataSize=78 B heapSize=488 B 2023-09-04 06:03:58,575 INFO [RS:0;jenkins-hbase4:35067] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-09-04 06:03:58,576 INFO [RS:0;jenkins-hbase4:35067] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-09-04 06:03:58,576 INFO [RS:0;jenkins-hbase4:35067] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-09-04 06:03:58,576 INFO [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-09-04 06:03:58,581 INFO [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-09-04 06:03:58,581 DEBUG [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(1478): Online Regions={b543154e41d822c029ea4a240097fa55=hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55., 1588230740=hbase:meta,,1.1588230740} 2023-09-04 06:03:58,581 DEBUG [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(1504): Waiting on 1588230740, b543154e41d822c029ea4a240097fa55 2023-09-04 06:03:58,581 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 06:03:58,581 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 06:03:58,581 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 06:03:58,582 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 06:03:58,582 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 06:03:58,582 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-09-04 06:03:58,605 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/namespace/b543154e41d822c029ea4a240097fa55/.tmp/info/e8bf50f26b604d6fad9a79297cca6bec 2023-09-04 06:03:58,609 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/.tmp/info/1b56475ff06d4b229daae4d60b499bf3 2023-09-04 06:03:58,615 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/namespace/b543154e41d822c029ea4a240097fa55/.tmp/info/e8bf50f26b604d6fad9a79297cca6bec as hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/namespace/b543154e41d822c029ea4a240097fa55/info/e8bf50f26b604d6fad9a79297cca6bec 2023-09-04 06:03:58,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/namespace/b543154e41d822c029ea4a240097fa55/info/e8bf50f26b604d6fad9a79297cca6bec, entries=2, sequenceid=6, filesize=4.8 K 2023-09-04 06:03:58,623 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for b543154e41d822c029ea4a240097fa55 in 47ms, sequenceid=6, compaction requested=false 2023-09-04 06:03:58,623 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-09-04 06:03:58,633 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/.tmp/table/83579434f33b4012a62b37f03ea18d2b 2023-09-04 06:03:58,634 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/namespace/b543154e41d822c029ea4a240097fa55/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-09-04 06:03:58,635 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55. 2023-09-04 06:03:58,635 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b543154e41d822c029ea4a240097fa55: 2023-09-04 06:03:58,635 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693807438044.b543154e41d822c029ea4a240097fa55. 2023-09-04 06:03:58,639 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/.tmp/info/1b56475ff06d4b229daae4d60b499bf3 as hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/info/1b56475ff06d4b229daae4d60b499bf3 2023-09-04 06:03:58,644 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/info/1b56475ff06d4b229daae4d60b499bf3, entries=10, sequenceid=9, filesize=5.9 K 2023-09-04 06:03:58,645 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/.tmp/table/83579434f33b4012a62b37f03ea18d2b as hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/table/83579434f33b4012a62b37f03ea18d2b 2023-09-04 06:03:58,650 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/table/83579434f33b4012a62b37f03ea18d2b, entries=2, sequenceid=9, filesize=4.7 K 2023-09-04 06:03:58,651 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 69ms, sequenceid=9, compaction requested=false 2023-09-04 06:03:58,651 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-09-04 06:03:58,657 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-09-04 06:03:58,658 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-04 06:03:58,658 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 06:03:58,658 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 06:03:58,658 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-09-04 06:03:58,739 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-09-04 06:03:58,739 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-09-04 06:03:58,781 INFO [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,35067,1693807437485; all regions closed. 2023-09-04 06:03:58,786 DEBUG [RS:0;jenkins-hbase4:35067] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/oldWALs 2023-09-04 06:03:58,787 INFO [RS:0;jenkins-hbase4:35067] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35067%2C1693807437485.meta:.meta(num 1693807437989) 2023-09-04 06:03:58,793 DEBUG [RS:0;jenkins-hbase4:35067] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/oldWALs 2023-09-04 06:03:58,793 INFO [RS:0;jenkins-hbase4:35067] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35067%2C1693807437485:(num 1693807437867) 2023-09-04 06:03:58,793 DEBUG [RS:0;jenkins-hbase4:35067] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:03:58,793 INFO [RS:0;jenkins-hbase4:35067] regionserver.LeaseManager(133): Closed leases 2023-09-04 06:03:58,794 INFO [RS:0;jenkins-hbase4:35067] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-09-04 06:03:58,794 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 06:03:58,794 INFO [RS:0;jenkins-hbase4:35067] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35067 2023-09-04 06:03:58,798 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): regionserver:35067-0x10075899e180001, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,35067,1693807437485 2023-09-04 06:03:58,798 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 06:03:58,798 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): regionserver:35067-0x10075899e180001, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 06:03:58,800 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,35067,1693807437485] 2023-09-04 06:03:58,800 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,35067,1693807437485; numProcessing=1 2023-09-04 06:03:58,801 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,35067,1693807437485 already deleted, retry=false 2023-09-04 06:03:58,801 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,35067,1693807437485 expired; onlineServers=0 2023-09-04 06:03:58,802 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40313,1693807437443' ***** 2023-09-04 06:03:58,802 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-09-04 06:03:58,802 DEBUG [M:0;jenkins-hbase4:40313] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@267da1fc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 06:03:58,802 INFO [M:0;jenkins-hbase4:40313] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40313,1693807437443 2023-09-04 06:03:58,802 INFO [M:0;jenkins-hbase4:40313] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40313,1693807437443; all regions closed. 2023-09-04 06:03:58,802 DEBUG [M:0;jenkins-hbase4:40313] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 06:03:58,802 DEBUG [M:0;jenkins-hbase4:40313] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-09-04 06:03:58,802 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-09-04 06:03:58,802 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693807437638] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693807437638,5,FailOnTimeoutGroup] 2023-09-04 06:03:58,802 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693807437639] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693807437639,5,FailOnTimeoutGroup] 2023-09-04 06:03:58,802 DEBUG [M:0;jenkins-hbase4:40313] cleaner.HFileCleaner(317): Stopping file delete threads 2023-09-04 06:03:58,803 INFO [M:0;jenkins-hbase4:40313] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-09-04 06:03:58,803 INFO [M:0;jenkins-hbase4:40313] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-09-04 06:03:58,804 INFO [M:0;jenkins-hbase4:40313] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-09-04 06:03:58,804 DEBUG [M:0;jenkins-hbase4:40313] master.HMaster(1512): Stopping service threads 2023-09-04 06:03:58,804 INFO [M:0;jenkins-hbase4:40313] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-09-04 06:03:58,804 ERROR [M:0;jenkins-hbase4:40313] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] 2023-09-04 06:03:58,804 INFO [M:0;jenkins-hbase4:40313] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-09-04 06:03:58,804 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-09-04 06:03:58,804 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-09-04 06:03:58,804 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 06:03:58,805 DEBUG [M:0;jenkins-hbase4:40313] zookeeper.ZKUtil(398): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-09-04 06:03:58,805 WARN [M:0;jenkins-hbase4:40313] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-09-04 06:03:58,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 06:03:58,805 INFO [M:0;jenkins-hbase4:40313] assignment.AssignmentManager(315): Stopping assignment manager 2023-09-04 06:03:58,805 INFO [M:0;jenkins-hbase4:40313] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-09-04 06:03:58,806 DEBUG [M:0;jenkins-hbase4:40313] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 06:03:58,806 INFO [M:0;jenkins-hbase4:40313] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:03:58,806 DEBUG [M:0;jenkins-hbase4:40313] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:03:58,806 DEBUG [M:0;jenkins-hbase4:40313] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 06:03:58,806 DEBUG [M:0;jenkins-hbase4:40313] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:03:58,806 INFO [M:0;jenkins-hbase4:40313] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.07 KB heapSize=29.55 KB 2023-09-04 06:03:58,816 INFO [M:0;jenkins-hbase4:40313] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.07 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3cbcdc68e4814a278ce42f175d8a4b40 2023-09-04 06:03:58,822 DEBUG [M:0;jenkins-hbase4:40313] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3cbcdc68e4814a278ce42f175d8a4b40 as hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3cbcdc68e4814a278ce42f175d8a4b40 2023-09-04 06:03:58,827 INFO [M:0;jenkins-hbase4:40313] regionserver.HStore(1080): Added hdfs://localhost:38345/user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3cbcdc68e4814a278ce42f175d8a4b40, entries=8, sequenceid=66, filesize=6.3 K 2023-09-04 06:03:58,828 INFO [M:0;jenkins-hbase4:40313] regionserver.HRegion(2948): Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 22ms, sequenceid=66, compaction requested=false 2023-09-04 06:03:58,829 INFO [M:0;jenkins-hbase4:40313] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 06:03:58,829 DEBUG [M:0;jenkins-hbase4:40313] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 06:03:58,832 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/dbc605f3-ebfd-40ad-4557-6cca95f064ef/MasterData/WALs/jenkins-hbase4.apache.org,40313,1693807437443/jenkins-hbase4.apache.org%2C40313%2C1693807437443.1693807437562 not finished, retry = 0 2023-09-04 06:03:58,933 INFO [M:0;jenkins-hbase4:40313] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-09-04 06:03:58,933 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 06:03:58,934 INFO [M:0;jenkins-hbase4:40313] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40313 2023-09-04 06:03:58,937 DEBUG [M:0;jenkins-hbase4:40313] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,40313,1693807437443 already deleted, retry=false 2023-09-04 06:03:58,974 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): regionserver:35067-0x10075899e180001, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:03:58,974 INFO [RS:0;jenkins-hbase4:35067] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,35067,1693807437485; zookeeper connection closed. 2023-09-04 06:03:58,974 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): regionserver:35067-0x10075899e180001, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:03:58,975 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7c957fd8] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7c957fd8 2023-09-04 06:03:58,975 INFO [Listener at localhost/44821] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-09-04 06:03:59,075 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:03:59,075 INFO [M:0;jenkins-hbase4:40313] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40313,1693807437443; zookeeper connection closed. 2023-09-04 06:03:59,075 DEBUG [Listener at localhost/44821-EventThread] zookeeper.ZKWatcher(600): master:40313-0x10075899e180000, quorum=127.0.0.1:54525, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 06:03:59,076 WARN [Listener at localhost/44821] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 06:03:59,080 INFO [Listener at localhost/44821] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:03:59,183 WARN [BP-699695109-172.31.14.131-1693807436887 heartbeating to localhost/127.0.0.1:38345] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 06:03:59,184 WARN [BP-699695109-172.31.14.131-1693807436887 heartbeating to localhost/127.0.0.1:38345] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-699695109-172.31.14.131-1693807436887 (Datanode Uuid 34fed369-8aa5-4aef-a3ae-81265950af3f) service to localhost/127.0.0.1:38345 2023-09-04 06:03:59,184 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/cluster_84f50541-547e-f103-32a1-70596c6f7281/dfs/data/data3/current/BP-699695109-172.31.14.131-1693807436887] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:03:59,185 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/cluster_84f50541-547e-f103-32a1-70596c6f7281/dfs/data/data4/current/BP-699695109-172.31.14.131-1693807436887] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:03:59,185 WARN [Listener at localhost/44821] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 06:03:59,188 INFO [Listener at localhost/44821] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:03:59,291 WARN [BP-699695109-172.31.14.131-1693807436887 heartbeating to localhost/127.0.0.1:38345] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 06:03:59,291 WARN [BP-699695109-172.31.14.131-1693807436887 heartbeating to localhost/127.0.0.1:38345] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-699695109-172.31.14.131-1693807436887 (Datanode Uuid 4d990609-f9da-404b-abf3-715efcc4d246) service to localhost/127.0.0.1:38345 2023-09-04 06:03:59,292 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/cluster_84f50541-547e-f103-32a1-70596c6f7281/dfs/data/data1/current/BP-699695109-172.31.14.131-1693807436887] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:03:59,293 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/62bd8a63-94c4-2edb-fc7d-5c26667e236f/cluster_84f50541-547e-f103-32a1-70596c6f7281/dfs/data/data2/current/BP-699695109-172.31.14.131-1693807436887] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 06:03:59,304 INFO [Listener at localhost/44821] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 06:03:59,416 INFO [Listener at localhost/44821] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-09-04 06:03:59,429 INFO [Listener at localhost/44821] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-09-04 06:03:59,439 INFO [Listener at localhost/44821] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=101 (was 77) - Thread LEAK? -, OpenFileDescriptor=493 (was 466) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=160 (was 160), ProcessCount=164 (was 164), AvailableMemoryMB=9172 (was 9179)